Summarising results from predictNMB

library(predictNMB)
library(parallel)
library(ggplot2)
library(flextable)
set.seed(42)

This vignette is purely about how to use the autoplot() method and summary() to visualise and summarise the simulations made using {predictNMB}. For an introduction to {predictNMB}, please see the introductory vignette.

Firstly, as an example case, we will prepare our sampling functions and run screen_simulation_inputs().

get_nmb_sampler_training <- get_nmb_sampler(
  wtp = 28033,
  qalys_lost = function() rnorm(n = 1, mean = 0.0036, sd = 0.0005),
  high_risk_group_treatment_cost = function() rnorm(n = 1, mean = 20, sd = 3),
  high_risk_group_treatment_effect = function() rbeta(n = 1, shape1 = 40, shape2 = 60),
  use_expected_values = TRUE
)

get_nmb_sampler_evaluation <- get_nmb_sampler(
  wtp = 28033,
  qalys_lost = function() rnorm(n = 1, mean = 0.0036, sd = 0.0005),
  high_risk_group_treatment_cost = function() rnorm(n = 1, mean = 20, sd = 3),
  high_risk_group_treatment_effect = function() rbeta(n = 1, shape1 = 40, shape2 = 60)
)
cl <- makeCluster(2)
sim_screen_obj <- screen_simulation_inputs(
  n_sims = 500,
  n_valid = 10000,
  sim_auc = seq(0.7, 0.95, 0.05),
  event_rate = c(0.1, 0.2),
  fx_nmb_training = get_nmb_sampler_training,
  fx_nmb_evaluation = get_nmb_sampler_evaluation,
  cutpoint_methods = c("all", "none", "youden", "value_optimising"),
  cl = cl
)
stopCluster(cl)

Making plots with predictNMB

Plotting the results from screen_simulation_inputs()

Choosing the x-axis variable and other constants

In this simulation screen, we vary both the event rate and the model discrimination (sim_AUC). There are many ways that we could visualise the data. The autoplot() function allows us to make some basic plots to compare the impact of different cutpoint methods on Net Monetary Benefit (NMB) and another variable of our choice.

In this case, we can visualise the impact on NMB for different methods across varying levels of sim_auc or event_rate. We control this with the x_axis_var argument.

autoplot(sim_screen_obj, x_axis_var = "sim_auc")
#> 
#> 
#> Varying simulation inputs, other than sim_auc, are being held constant:
#> event_rate: 0.1

To avoid the overlap of points in this second plot, which visualises NMB at varying event rates, we can specify the dodge_width to be non-zero.

autoplot(sim_screen_obj, x_axis_var = "event_rate", dodge_width = 0.002)
#> 
#> 
#> Varying simulation inputs, other than event_rate, are being held constant:
#> sim_auc: 0.7

For these plots, one of the screened inputs will be the x-axis variable, but the other will only be displayed at a single level. For example, if we are looking at outcomes over a range of AUC values, the prevalence will be fixed. The default setting for this second input will assume the first level, so when we visualise the change in NMB specifying sim_auc as the x-axis variable, we only observe this for the case where event_rate = 0.1. We can choose to select another level with the constants argument. This argument expects a named list containing the values to keep for the screened inputs which are not shown on the x-axis.

autoplot(sim_screen_obj, x_axis_var = "sim_auc", constants = list(event_rate = 0.1))
#> 
#> 
#> Varying simulation inputs, other than sim_auc, are being held constant:
#> event_rate: 0.1

autoplot(sim_screen_obj, x_axis_var = "sim_auc", constants = list(event_rate = 0.2))
#> 
#> 
#> Varying simulation inputs, other than sim_auc, are being held constant:
#> event_rate: 0.2

We see both a change to the plot as well as the message produced when the plot is made.

Choosing a y-axis variable

There are three options for the y-axis. The default is the NMB, but you can also visualise the Incremental Net Monetary Benefit (INB) and the selected cutpoints. These are controlled by the what argument, which can be any of c("nmb", "inb", "cutpoints"). If a vector is used, only the first value will be selected. If you choose to visualise the INB, you must list your chosen reference strategy for the calculation in the inb_ref_col. In this case, we use treat-all ("all").

autoplot(sim_screen_obj, what = "nmb")

autoplot(sim_screen_obj, what = "inb", inb_ref_col = "all")

autoplot(sim_screen_obj, what = "cutpoints")

Selecting what to show on the plot

The plots show the median (the dot), the 95% confidence interval (thick vertical lines), the range (thin vertical lines), and the lines between the points by default. These can each be shown or hidden independently, and the width of the confidence interval can be controlled using the plot_conf_level argument.

autoplot(sim_screen_obj)

autoplot(sim_screen_obj, plot_range = FALSE)

autoplot(sim_screen_obj, plot_conf_level = FALSE)

autoplot(sim_screen_obj, plot_conf_level = FALSE, plot_range = FALSE)