15 Risk of Bias Plots

by Luke A. McGuinness


Please cite this chapter as:

McGuinness, L. A. (2021). Risk of Bias Plots. In Harrer, M., Cuijpers, P., Furukawa, T.A., & Ebert, D.D., Doing Meta-Analysis with R: A Hands-On Guide (online version). https://bookdown.org/MathiasHarrer/Doing_Meta_Analysis_in_R/rob-plots.html.


I n this chapter, we will describe how to create risk of bias plots in R, using the {robvis} package.


15.1 Introduction


As part of a systematic review and meta-analysis, you may also want to examine the internal validity (risk of bias) of included studies using the relevant domain-based risk of bias assessment tool, and present the results of this assessment in a graphical format.

The Cochrane Handbook recommends two types of figure: a summary barplot figure showing the proportion of studies with a given risk of bias judgement within each domain, and a traffic light plot which presents the domain level judgments for each study.

However, the options available to researchers when creating these figures are limited. While RevMan has the functionality to create the plots, many researchers do not use it to conduct their systematic review and so copying the relevant data into the system is an inefficient solution.

Similarly, producing the graphs by hand, using software such as MS PowerPoint, is time consuming and means the figures have to manually updated if changes are needed. Additionally, journals usually require figures to be of publication quality (above ~300-400dpi), which can be hard to achieve when exporting the risk of bias figures from RevMan or creating them by hand.

Example RevMan output.

Figure 15.1: Example RevMan output.

To avoid all of this, you can now easily plot the risk of bias figures yourself within R Studio, using the {robvis} package (McGuinness and Higgins 2020; McGuinness 2019) which provides functions to convert a risk of bias assessment summary table into a summary plot or a traffic-light plot.


15.1.1 Load {robvis}


Assuming that you have already installed the {dmetar} package (see Chapter 2.3), load the {robvis} package using:


15.1.2 Importing Your Risk of Bias Summary Table Data


To produce our plots, we first have to import the results of our risk of bias assessment from Excel into R. Please note that {robvis} expects certain facts about the data you provide it, so be sure to follow the guidance below when setting up your table in Excel:

  1. The first column is labelled “Study” and contains the study identifier (e.g. Anthony et al, 2019)
  2. The second-to-last column is labelled “Overall” and contains the overall risk-of-bias judgments
  3. The last column is labelled “Weight” and contains some measure of study precision e.g. the weight assigned to each study in the meta-analysis, or if no meta-analysis was performed, the sample size of each study). See Chapter 4.1.1 for more details.
  4. All other columns contain the results of the risk-of bias assessment for a specific domain.

To elaborate on the above guidance, consider as an example the ROB2 tool which has 5 domains. The resulting data set that {robvis} would expect for this tool would have 8 columns:

  • Column 1: Study identifier
  • Column 2-6: One RoB2 domain per column
  • Column 7: Overall risk-of-bias judgments
  • Column 8: Weight.

In Excel, this risk of bias summary table would look like this:

Column Names

For three of the four tool templates (ROB2, ROBINS-I, QUADAS-2), what you name the columns containing the domain-level judgments is not important, as the templates within robvis will relabel each domain with the correct tool-specific heading.

Once you have saved the table you created in Excel to the working directory as a comma-separated-file (e.g. “robdata.csv”), you can either read the file into R programmatically using the command below or via the “import assistant” method as described in Chapter 2.4.

my_rob_data <- read.csv("robdata.csv", header = TRUE)


15.1.3 Templates


{robvis} produces the risk of bias figures by using the data you provide to populate a template figure specific to the risk of bias assessment tool you used. At present, {robvis} contains templates for the following three tools:

  • ROB2, the new Cochrane risk of bias tool for randomized controlled trials;
  • ROBINS-I, the Risk of Bias In Non-randomized Studies - of Interventions tool;
  • QUADAS-2, the Quality and Applicability of Diagnostic Accuracy Studies, Version 2.

{robvis} also contains a special generic template, labeled as ROB1. Designed for use with the original Cochrane risk of bias tool for randomized controlled trials, it can also be used to visualize the results of assessments performed with other domain-based tools not included in the list above. See Section XXXXXXXX for more information on the additional steps required when using this template.


15.1.4 Example Data Sets


The {robvis} package contains an example data set for each template outlined above. These are stored in the following objects:

  • data_rob2: Example data for the ROB2 tool
  • data_robins: Example data for the ROBINS-I tool
  • data_quadas: Example data for the QUADAS-2 tool
  • data_rob1: Example data for the RoB-1 tool.

You can explore these data sets using the glimpse function (see Chapter 2.5.1). For example, once you have loaded the package using library(robvis), viewing the ROBINS-I example data set can be achieved by running the following command:

glimpse(data_robins)
## Rows: 12
## Columns: 10
## $ Study   <fct> Study 1, Study 2, Study 3, Study 4, Study 5, Study 6, Study 7…
## $ D1      <fct> Critical, Moderate, Moderate, Low, Serious, Critical, Critica…
## $ D2      <fct> Low, Low, Low, Low, Serious, Serious, Moderate, Moderate, Low…
## $ D3      <fct> Critical, Low, Moderate, Serious, Low, Moderate, Moderate, Lo…
## $ D4      <fct> Critical, Critical, Critical, Critical, Low, Critical, Seriou…
## $ D5      <fct> Low, Low, Critical, Moderate, Moderate, Critical, Critical, L…
## $ D6      <fct> Low, Moderate, Low, Low, Low, Moderate, Serious, Low, Serious…
## $ D7      <fct> Serious, Low, Serious, Critical, Moderate, Serious, Serious, …
## $ Overall <fct> Critical, Low, Serious, Low, Serious, Serious, Moderate, Mode…
## $ Weight  <dbl> 33.3333333, 33.3333333, 0.1428571, 9.0909091, 12.5000000, 25.…

These example data sets are used to create the plots presented through the remainder of this guide.


15.2 Summary Plots


15.2.1 Basics


Once we have successfully imported the risk of bias summary table into R, creating the risk of bias figures is quite straightforward.

To get started, a simple weighted summary bar plot using the ROB2 example data set (data_rob2) is created by running the following code:

rob_summary(data = data_rob2, 
            tool = "ROB2")


15.2.2 Modifying the Plot


The rob_summary function has the following parameters:

  • data. A data frame containing summary (domain) level risk-of-bias assessments, with the first column containing the study details, the second column containing the first domain of your assessments, and the final column containing a weight to assign to each study. The function assumes that the data includes a column for overall risk-of-bias. For example, a ROB2.0 dataset would have 8 columns (1 for study details, 5 for domain level judgments, 1 for overall judgments, and 1 for weights, in that order).
  • tool. The risk of bias assessment tool used. RoB2.0 ("ROB2"), "ROBINS-I", and "QUADAS-2" are currently supported.
  • overall. An option to include an additional bar for overall risk-of-bias in the figure. Default is FALSE.
  • weighted. An option to specify whether weights should be used in the bar plot. Default is TRUE, in line with current Cochrane Collaboration guidance.
  • colour. An argument to specify the colour scheme for the plot. Default is "cochrane", which used the ubiquitous Cochrane colours, while a preset option for a colour-blind friendly palette is also available (colour = "colourblind").
  • quiet. A logical option to quietly produce the plot without displaying it. Default is FALSE.

Examples of the functionality of each argument are described below.


15.2.2.1 Tool


An argument to define the tool template you wish to use. In the example above, the ROB2 template is used. The two other primary templates - the ROBINS-I and QUADAS-2 templates - are demonstrated below:

rob_summary(data = data_robins, 
            tool = "ROBINS-I")
rob_summary(data = data_quadas, 
            tool = "QUADAS-2")


15.2.2.2 Overall


By default, an additional bar representing the overall risk of bias judgments is not included in the plot. If you would like to include this, set overall = TRUE. For example:

rob_summary(data = data_rob2, 
            tool = "ROB2", 
            overall = TRUE)


15.2.2.3 Weighted or Unweighted Bar Plots


By default, the bar plot is weighted by some measure of study precision, so that the bar plot shows the proportion of information rather than the proportion of studies that is at a particular risk of bias. This approach is in line with the Cochrane Handbook.

You can turn off this option by setting weighted = FALSE to create an unweighted bar plot. For example, compare the following two plots:

rob_summary(data = data_rob2, 
            tool = "ROB2")
rob_summary(data = data_rob2, 
            tool = "ROB2",
            weighted = FALSE)


15.2.2.4 Colour Scheme


British English Spelling

Please note the non-US English spelling of colour!

The colour argument of both plotting functions allows users to select from two predefined colour schemes, "cochrane" (default) or "colourblind", or to define their own palette by providing a vector of hex codes. For example, to use the predefined "colourblind" palette:

rob_summary(data = data_rob2, 
            tool = "ROB2", 
            colour = "colourblind")

And to define your own colour scheme:

rob_summary(data = data_rob2, 
            tool = "ROB2", 
            colour = c("#f442c8","#bef441","#000000"))

When defining your own colour scheme, you must ensure that the number of discrete judgments (e.g. “Low”, “Moderate”, “High”, “Critical”) and the number of colours specified are the same. Additionally, colours must be specified in order of ascending risk-of-bias (e.g. “Low” to “Critical”), with the first hex corresponding to “Low” risk of bias.


15.3 Traffic Light Plots


Frequently, researchers will want to present the risk of bias in each domain for each study assessed. The resulting plots are commonly called traffic light plots, and can be produced with {robvis} via the rob_traffic_light function.


15.3.1 Basics


To get started, a traffic light plot using the ROB2 example dataset (data_rob2) is created by running the following code:

rob_traffic_light(data = data_rob2, 
                  tool = "ROB2")


15.3.2 Modifying the Plot


The rob_summary function has the following parameters:

  • data. A data frame containing summary (domain) level risk-of-bias assessments, with the first column containing the study details, the second column containing the first domain of your assessments, and the final column containing a weight to assign to each study. The function assumes that the data includes a column for overall risk-of-bias. For example, a ROB2.0 data set would have 8 columns (1 for study details, 5 for domain level judgments, 1 for overall judgments, and 1 for weights, in that order).
  • tool. The risk of bias assessment tool used. RoB2.0 ("ROB2"), "ROBINS-I", and "QUADAS-2" are currently supported.
  • colour. An argument to specify the colour scheme for the plot. Default is "cochrane" which used the ubiquitous Cochrane colours, while a preset option for a colour-blind friendly palette is also available ("colourblind").
  • psize. An option to change the size of the “traffic light” points. Default is 20.
  • quiet. A logical option to quietly produce the plot without displaying it. Default is FALSE.


15.3.2.0.1 Tool

An argument to define the tool template you wish to use. The ROB2 template is demonstrated and the two other primary templates - the ROBINS-I and QUADAS-2 templates - are displayed below:

rob_traffic_light(data = data_robins, 
                  tool = "ROBINS-I")
rob_traffic_light(data = data_quadas, 
                  tool = "QUADAS-2")


15.3.2.0.2 Colour Scheme

British English Spelling

Please note the non-US English spelling of colour!

The colour argument of both plotting functions allows users to select from two predefined colour schemes, "cochrane" (default) or "colourblind", or to define their own palette by providing a vector of hex codes.

For example, to use the predefined "colourblind" palette:

rob_traffic_light(data = data_rob2, 
                  tool = "ROB2", 
                  colour = "colourblind")

And to define your own colour scheme:

rob_traffic_light(data = data_rob2, 
                  tool = "ROB2", 
                  colour = c("#f442c8","#bef441","#000000"))

When defining your own colour scheme, you must ensure that the number of discrete judgments (e.g. “Low”, “Moderate”, “High”, “Critical”) and the number of colours specified are the same. Additionally, colours must be specified in order of ascending risk-of-bias (e.g. “Low” to “Critical”), with the first hex corresponding to “Low” risk of bias.


15.3.2.0.3 Point Size

Occasionally, when a large number of risk of bias assessment have been performed, the resulting traffic light plot may be too long to be useful. Users can address this by modifying the psize argument of the rob_traffic_light function to a smaller number (default is 20). For example:

# Create bigger dataset (18 studies)
new_rob2_data <- rbind(data_rob2, data_rob2)
new_rob2_data$Study <- paste("Study", seq(1:length(new_rob2_data$Study))) 

# Plot bigger dataset, reducing the psize argument from 20 to 8
rob_traffic_light(data = new_rob2_data, 
                  tool = "ROB2", 
                  psize = 8)


15.3.2.0.4 The “ROB1” Generic Template

\[\tag*{$\blacksquare$}\]