Chapter 10 The randomized controlled trial as a method for controlling biases

10.1 Learning objectives

By the end of this chapter, you will be able to:

  • Explain which biases from previous chapters are adequately controlled in a randomized controlled trial
  • Understand how reporting standards can help ensure study quality

10.2 The ingredients of a RCT

The randomized controlled trial (RCT) is regarded by many as the gold standard method for evaluating interventions. In Chapter 15 we will discuss some of the limitations of this approach that can make it less than ideal for evaluating certain kinds of non-medical interventions. But in this chapter we’ll look at the ingredients of a RCT that make it such a well-regarded method, and introduce the basic methods that can be used to analyse the results.

A RCT is effective simply because it is designed to counteract all of the systematic biases that were covered in previous Chapters.

Table 10.1: How the RCT design deals with threats to study validity
Biases Remedies
Spontaneous improvement Control group
Practice effects Control group
Regression to the mean Control group
Noisy data (1) Strict selection criteria for participants
Noisy data (2) Outcomes with low measurement error
Selection bias Random assignment to intervention
Placebo effects Participant unaware of assignment
Experimenter bias (1) Experimenter unaware of assignment
Experimenter bias (2) Strictly specified protocol
Biased drop-outs Intention-to-treat/Instrumental variable analysis
Low power A priori power analysis
False positives due to p-hacking Registration of trial protocol

We cannot prevent changes in outcomes over time that arise for reasons other than intervention (Chapter 4), but if we include a control group we can estimate and correct for their influence. Noisy data in general arises either because of heterogenous participants, or because of unreliable measures: in a good RCT there will be strict selection criteria for participants, and careful choice of outcome measures to be psychometrically sound (see Chapter 3). Randomization of participants to intervention and control groups avoids bias caused either by participants’ self-selection of intervention group, or researchers determining who gets what treatment. In addition, as noted in Chapter 4 and Chapter 8, where feasible, both participants and experimenters are kept unaware of who is in which treatment group, giving a double blind RCT.

10.3 Primary and secondary outcomes

In Chapter 14, we will explain problems caused by p-hacking, where many outcome measures are included but only the “significant” ones are reported. This problem has been recognized in the context of clinical trials for many years, which is why clinical trial protocols are usually registered specifying a primary outcome measure of interest: indeed, as is discussed further in Chapter 22, many journals will not accept a trial for publication unless it has been registered on a site such as https://clinicaltrials.gov/. Note, however, that, as will be discussed in Chapter 14, multiple outcomes may increase the statistical power of a study, and are not a problem if the statistical analysis handles the multiplicity correctly. Secondary outcome measures can also be specified, but reporting of analyses relating to these outcomes should make it clear that they are much more exploratory. In principle, this should limit the amount of data dredging for a result that is only loosely related to the anticipated intervention effect.

10.4 Reporting standards

RCTs have become such a bedrock of medical research that standards for reporting them have been developed. In Chapter 9 we saw the CONSORT flowchart, which is a useful way of documenting the flow of participants through a trial. CONSORT stands for Consolidated Standards of Reporting Trials, which are endorsed by many medical journals. Indeed, if you plan to publish an intervention study in one of those journals, you are likely to be required to show you have followed the guidelines. The relevant information is available on the ‘Enhancing the QUAlity and Transparency Of health Research’ (EQUATOR) network website (http://www.equator-network.org). The EQUATOR network site covers not only RCTs but also the full spectrum of guidelines of many types of clinical research designs.

For someone starting out planning a trial, it is worth reading the CONSORT Explanation and Elaboration document (Moher et al., 2010), which gives the rationale behind different aspects of the CONSORT guidelines. This may seem rather daunting to beginners, as it mentions more complex trial designs as well as a standard RCT comparing intervention and control groups, and it assumes a degree of statistical expertise. It is nevertheless worth studying, as adherence to CONSORT guidelines is seen as a marker of study quality, and it is much easier to conform to their recommendations if a study is planned with the guidelines in mind, rather than if the guidelines are only consulted after the study is done.

10.5 Check your understanding

  1. The CONSORT Explanation and Elaboration document (Moher et al., 2010) notes the importance of specifying details of study design and implementation, including the following:
  • Eligibility criteria for participants
  • Settings and locations where the data were collected
  • The interventions for each group with sufficient details to allow replication, including how and when they were actually administered

Find a published intervention study in your area of interest, and compare reporting of these features with the description of what is required by Moher et al. (2010).

  1. In Chapter 15 we consider drawbacks of the RCT design. Before you read that chapter, see if you can anticipate the issues that we consider in our evaluation. In what circumstances would an RCT be impractical or inappropriate?

References

Moher, D., Hopewell, S., Schulz, K. F., Montori, V., Gøtzsche, P. C., Devereaux, P. J., Elbourne, D., Egger, M., & Altman, D. G. (2010). CONSORT 2010 explanation and elaboration: Updated guidelines for reporting parallel group randomised trials. British Medical Journal (Clinical Research Ed.), 340, c869. https://doi.org/10.1136/bmj.c869