Chapter 11 Network Meta-Analysis

Often when doing meta-analysis on the effectiveness of certain interventions, we are less interested if one particular intervention is effective (e.g., because it is quite well established that the intervention can be efficacious), but weather one intervention is more or less effective than another type of intervention for some condition. Yet, once we are interested in head-to-head comparisons between two treatments, we often face the problem that only very few, if any, randomized controlled trials have compared the effects of two interventions directly. This makes it very hard, if not impossible for us to conduct conventional meta-analyses to answer questions on the comparative effects of two or more interventions for one indication or outcome (e.g., different types of psychotherapy for major depression).

Nevertheless, while direct comparisons between two or more interventions may often not exist, it is often the case that the interventions were evaluated in separate randomized controlled trials in which the intervention effects were compared to similar control groups (e.g., waitlist control groups, or placebos). This means that we do have indirect comparisons of the effects of different interventions, because they were compared to the same control condition. Multiple-treatments meta-analysis (MTM) is an extension of conventional meta-analysis which allows us to incorporate indirect comparisons, and thus the simultaneous analysis of several interventions.

These meta-analysis methods are also referred to as network meta-analyses, as such methods allow for multiple interventions comparisons to be integrated into our analysis, which can be formalized as a “network” of comparisons.



The idea behind network meta-analysis

Let’s assume you have the results of two randomized controlled trials. One trial evaluated the effect of cognitive behavioral therapy (CBT) for depression to a control group. The second trial evaluted the effects of short-term psychodynamic therapy (STPP) on depression compared to control. We know the effect size \(\hat\theta\) of both interventions which was found in the trial compared to control at post-test. These studies produce indirect evidence for the comparative effect of CBT versus STPP (Schwarzer, Carpenter, and Rücker 2015):

\[\hat\theta_{CBT vs. STPP}^{indirect}=\hat\theta_{CBT vs. Control}^{direct} -\hat\theta_{STPP vs.Control}^{direct} \]

On the other hand, it may also be the case that we found one study in which the effects of CBT were directly compared to the ones of STPP. We will denote this effect as \(\hat\theta_{CBT vs. STPP}^{direct}\). In network-meta-analysis, we want to integrate the direct as well as the indirect evidence to get the most precise effect estimate of the comparative effects.

According to Schwarzer et al. (Schwarzer, Carpenter, and Rücker 2015), there are two conditions which have to be met to conduct network meta-analyses:

  • The studies are independent
  • The effect sizes are consistent. This means that effect sizes of interventions comparisons we attain through direct evidence should be similar to the one we get from indirect evidence (e.g., \(\theta_{CBT vs. STPP}^{direct}=\theta_{CBT vs. STPP}^{indirect}\)). Inconsistency, on the other hand, is \(\theta_{CBT vs. STPP}^{direct}-\theta_{CBT vs. STPP}^{indirect0} \not= 0\). Assessing and dealing with inconsistency is highly important in network meta-analysis.

Below, you can see a simple first network of the comparisons between the control condition and the two interventions. We could also think of a network where some comparisons are simply not available, as is the case in the second network.

A simple network

Figure 11.1: A simple network

A simple network with one missing comparison

Figure 11.2: A simple network with one missing comparison

Work in progress

A full version of this Chapter will be available soon.

library(netmeta)
library(plyr)
netmetdata
TE seTE treat1 treat2 Author
1 -1.90 0.1414 MCT Waitlist DeFronzo1995
2 -0.82 0.0992 MCT Waitlist Lewin2007
4 -1.34 0.1435 CBT Waitlist Davidson2007
5 -1.10 0.1141 CBT Waitlist Wolffenbuttel1999
6 -1.30 0.1268 MBCT Waitlist Kipnes2001
7 -0.77 0.1078 CBT Waitlist Kerenyi2004
8 0.16 0.0849 MBCT MCT Hanefeld2004
9 0.10 0.1831 MBCT CBT Derosa2004
10 -1.30 0.1014 CBT Waitlist Baksi2004
11 -1.09 0.2263 CBT Waitlist Rosenstock2008
12 -1.50 0.1624 CBT Waitlist Zhu2003
13 -0.14 0.2239 CBT MCT Yang2003
14 -1.20 0.1436 CBT Placebo Vongthavaravat2002
15 -0.40 0.1549 Psychodynamic Placebo Oyama2008
16 -0.80 0.1432 Psychodynamic Waitlist Costa1997
17 -0.57 0.1291 System Waitlist Hermansen2007
18 -0.70 0.1273 Gestalt Waitlist Garber2008
19 -0.37 0.1184 MCT Placebo Alex1998
20 -0.74 0.1839 Psychoanalysis Waitlist Johnston1994
21 -1.41 0.2235 Psychoanalysis Waitlist Johnston1998a
22 0.00 0.2339 CBT MCT Kim2007
23 -0.68 0.2828 Psychoanalysis Waitlist Johnston1998b
24 -0.40 0.4356 MCT Waitlist Gonzalez-Ortiz2004
25 -0.23 0.3467 ACT Waitlist Stucci1996
26 -1.01 0.1366 ACT Waitlist Moulin2006
27 -1.20 0.3758 MCT Waitlist Ebert2018
28 -1.00 0.4669 Psychodynamic Waitlist Ebert2018
29 -0.20 0.3579 MCT Psychodynamic Ebert2018

Results

netmet <- netmeta(TE, seTE, treat1, treat2, studlab=paste(Author), data=netmetdata, sm="SMD",reference.group = "Waitlist")
## Warning: Note, treatments within a comparison have been re-sorted in
## increasing order.
netmet
## Original data (with adjusted standard errors for multi-arm studies):
## 
##                            treat1        treat2      TE   seTE seTE.adj
## DeFronzo1995                  MCT      Waitlist -1.9000 0.1414   0.1414
## Lewin2007                     MCT      Waitlist -0.8200 0.0992   0.0992
## Davidson2007                  CBT      Waitlist -1.3400 0.1435   0.1435
## Wolffenbuttel1999             CBT      Waitlist -1.1000 0.1141   0.1141
## Kipnes2001                   MBCT      Waitlist -1.3000 0.1268   0.1268
## Kerenyi2004                   CBT      Waitlist -0.7700 0.1078   0.1078
## Hanefeld2004                 MBCT           MCT  0.1600 0.0849   0.0849
## Derosa2004                    CBT          MBCT -0.1000 0.1831   0.1831
## Baksi2004                     CBT      Waitlist -1.3000 0.1014   0.1014
## Rosenstock2008                CBT      Waitlist -1.0900 0.2263   0.2263
## Zhu2003                       CBT      Waitlist -1.5000 0.1624   0.1624
## Yang2003                      CBT           MCT -0.1400 0.2239   0.2239
## Vongthavaravat2002            CBT       Placebo -1.2000 0.1436   0.1436
## Oyama2008                 Placebo Psychodynamic  0.4000 0.1549   0.1549
## Costa1997           Psychodynamic      Waitlist -0.8000 0.1432   0.1432
## Hermansen2007              System      Waitlist -0.5700 0.1291   0.1291
## Garber2008                Gestalt      Waitlist -0.7000 0.1273   0.1273
## Alex1998                      MCT       Placebo -0.3700 0.1184   0.1184
## Johnston1994       Psychoanalysis      Waitlist -0.7400 0.1839   0.1839
## Johnston1998a      Psychoanalysis      Waitlist -1.4100 0.2235   0.2235
## Kim2007                       CBT           MCT  0.0000 0.2339   0.2339
## Johnston1998b      Psychoanalysis      Waitlist -0.6800 0.2828   0.2828
## Gonzalez-Ortiz2004            MCT      Waitlist -0.4000 0.4356   0.4356
## Stucci1996                    ACT      Waitlist -0.2300 0.3467   0.3467
## Moulin2006                    ACT      Waitlist -1.0100 0.1366   0.1366
## Ebert2018                     MCT      Waitlist -1.2000 0.3758   0.4125
## Ebert2018           Psychodynamic      Waitlist -1.0000 0.4669   0.8242
## Ebert2018                     MCT Psychodynamic -0.2000 0.3579   0.3884
##                    narms multiarm
## DeFronzo1995           2         
## Lewin2007              2         
## Davidson2007           2         
## Wolffenbuttel1999      2         
## Kipnes2001             2         
## Kerenyi2004            2         
## Hanefeld2004           2         
## Derosa2004             2         
## Baksi2004              2         
## Rosenstock2008         2         
## Zhu2003                2         
## Yang2003               2         
## Vongthavaravat2002     2         
## Oyama2008              2         
## Costa1997              2         
## Hermansen2007          2         
## Garber2008             2         
## Alex1998               2         
## Johnston1994           2         
## Johnston1998a          2         
## Kim2007                2         
## Johnston1998b          2         
## Gonzalez-Ortiz2004     2         
## Stucci1996             2         
## Moulin2006             2         
## Ebert2018              3        *
## Ebert2018              3        *
## Ebert2018              3        *
## 
## Number of treatment arms (by study):
##                    narms
## Alex1998               2
## Baksi2004              2
## Costa1997              2
## Davidson2007           2
## DeFronzo1995           2
## Derosa2004             2
## Ebert2018              3
## Garber2008             2
## Gonzalez-Ortiz2004     2
## Hanefeld2004           2
## Hermansen2007          2
## Johnston1994           2
## Johnston1998a          2
## Johnston1998b          2
## Kerenyi2004            2
## Kim2007                2
## Kipnes2001             2
## Lewin2007              2
## Moulin2006             2
## Oyama2008              2
## Rosenstock2008         2
## Stucci1996             2
## Vongthavaravat2002     2
## Wolffenbuttel1999      2
## Yang2003               2
## Zhu2003                2
## 
## Results (fixed effect model):
## 
##                            treat1        treat2     SMD             95%-CI
## DeFronzo1995                  MCT      Waitlist -1.1141 [-1.2309; -0.9973]
## Lewin2007                     MCT      Waitlist -1.1141 [-1.2309; -0.9973]
## Davidson2007                  CBT      Waitlist -1.2018 [-1.2953; -1.1084]
## Wolffenbuttel1999             CBT      Waitlist -1.2018 [-1.2953; -1.1084]
## Kipnes2001                   MBCT      Waitlist -1.0664 [-1.2151; -0.9178]
## Kerenyi2004                   CBT      Waitlist -1.2018 [-1.2953; -1.1084]
## Hanefeld2004                 MBCT           MCT  0.0477 [-0.0891;  0.1845]
## Derosa2004                    CBT          MBCT -0.1354 [-0.2957;  0.0249]
## Baksi2004                     CBT      Waitlist -1.2018 [-1.2953; -1.1084]
## Rosenstock2008                CBT      Waitlist -1.2018 [-1.2953; -1.1084]
## Zhu2003                       CBT      Waitlist -1.2018 [-1.2953; -1.1084]
## Yang2003                      CBT           MCT -0.0877 [-0.2203;  0.0449]
## Vongthavaravat2002            CBT       Placebo -0.7623 [-0.9427; -0.5820]
## Oyama2008                 Placebo Psychodynamic  0.3879 [ 0.1662;  0.6095]
## Costa1997           Psychodynamic      Waitlist -0.8274 [-1.0401; -0.6147]
## Hermansen2007              System      Waitlist -0.5700 [-0.8230; -0.3170]
## Garber2008                Gestalt      Waitlist -0.7000 [-0.9495; -0.4505]
## Alex1998                      MCT       Placebo -0.6746 [-0.8482; -0.5011]
## Johnston1994       Psychoanalysis      Waitlist -0.9439 [-1.1927; -0.6952]
## Johnston1998a      Psychoanalysis      Waitlist -0.9439 [-1.1927; -0.6952]
## Kim2007                       CBT           MCT -0.0877 [-0.2203;  0.0449]
## Johnston1998b      Psychoanalysis      Waitlist -0.9439 [-1.1927; -0.6952]
## Gonzalez-Ortiz2004            MCT      Waitlist -1.1141 [-1.2309; -0.9973]
## Stucci1996                    ACT      Waitlist -0.9052 [-1.1543; -0.6561]
## Moulin2006                    ACT      Waitlist -0.9052 [-1.1543; -0.6561]
## Ebert2018                     MCT      Waitlist -1.1141 [-1.2309; -0.9973]
## Ebert2018           Psychodynamic      Waitlist -0.8274 [-1.0401; -0.6147]
## Ebert2018                     MCT Psychodynamic -0.2867 [-0.5113; -0.0622]
##                        Q leverage
## DeFronzo1995       30.89     0.18
## Lewin2007           8.79     0.36
## Davidson2007        0.93     0.11
## Wolffenbuttel1999   0.80     0.17
## Kipnes2001          3.39     0.36
## Kerenyi2004        16.05     0.20
## Hanefeld2004        1.75     0.68
## Derosa2004          0.04     0.20
## Baksi2004           0.94     0.22
## Rosenstock2008      0.24     0.04
## Zhu2003             3.37     0.09
## Yang2003            0.05     0.09
## Vongthavaravat2002  9.29     0.41
## Oyama2008           0.01     0.53
## Costa1997           0.04     0.57
## Hermansen2007       0.00     1.00
## Garber2008          0.00     1.00
## Alex1998            6.62     0.56
## Johnston1994        1.23     0.48
## Johnston1998a       4.35     0.32
## Kim2007             0.14     0.08
## Johnston1998b       0.87     0.20
## Gonzalez-Ortiz2004  2.69     0.02
## Stucci1996          3.79     0.13
## Moulin2006          0.59     0.87
## Ebert2018           0.04     0.02
## Ebert2018           0.04     0.02
## Ebert2018           0.05     0.09
## 
## Results (random effects model):
## 
##                            treat1        treat2     SMD             95%-CI
## DeFronzo1995                  MCT      Waitlist -1.1268 [-1.4291; -0.8244]
## Lewin2007                     MCT      Waitlist -1.1268 [-1.4291; -0.8244]
## Davidson2007                  CBT      Waitlist -1.2335 [-1.4839; -0.9830]
## Wolffenbuttel1999             CBT      Waitlist -1.2335 [-1.4839; -0.9830]
## Kipnes2001                   MBCT      Waitlist -1.1291 [-1.5596; -0.6986]
## Kerenyi2004                   CBT      Waitlist -1.2335 [-1.4839; -0.9830]
## Hanefeld2004                 MBCT           MCT -0.0023 [-0.4444;  0.4398]
## Derosa2004                    CBT          MBCT -0.1044 [-0.5435;  0.3347]
## Baksi2004                     CBT      Waitlist -1.2335 [-1.4839; -0.9830]
## Rosenstock2008                CBT      Waitlist -1.2335 [-1.4839; -0.9830]
## Zhu2003                       CBT      Waitlist -1.2335 [-1.4839; -0.9830]
## Yang2003                      CBT           MCT -0.1067 [-0.4304;  0.2170]
## Vongthavaravat2002            CBT       Placebo -0.8169 [-1.2817; -0.3521]
## Oyama2008                 Placebo Psychodynamic  0.4252 [-0.0951;  0.9456]
## Costa1997           Psychodynamic      Waitlist -0.8418 [-1.3236; -0.3600]
## Hermansen2007              System      Waitlist -0.5700 [-1.2640;  0.1240]
## Garber2008                Gestalt      Waitlist -0.7000 [-1.3927; -0.0073]
## Alex1998                      MCT       Placebo -0.7102 [-1.1713; -0.2491]
## Johnston1994       Psychoanalysis      Waitlist -0.9497 [-1.4040; -0.4955]
## Johnston1998a      Psychoanalysis      Waitlist -0.9497 [-1.4040; -0.4955]
## Kim2007                       CBT           MCT -0.1067 [-0.4304;  0.2170]
## Johnston1998b      Psychoanalysis      Waitlist -0.9497 [-1.4040; -0.4955]
## Gonzalez-Ortiz2004            MCT      Waitlist -1.1268 [-1.4291; -0.8244]
## Stucci1996                    ACT      Waitlist -0.7311 [-1.2918; -0.1705]
## Moulin2006                    ACT      Waitlist -0.7311 [-1.2918; -0.1705]
## Ebert2018                     MCT      Waitlist -1.1268 [-1.4291; -0.8244]
## Ebert2018           Psychodynamic      Waitlist -0.8418 [-1.3236; -0.3600]
## Ebert2018                     MCT Psychodynamic -0.2850 [-0.7908;  0.2208]
## 
## Number of studies: k = 26
## Number of treatments: n = 10
## Number of pairwise comparisons: m = 28
## Number of designs: d = 15
## 
## Fixed effects model
## 
## Treatment estimate (sm = 'SMD', comparison: other treatments vs 'Waitlist'):
##                    SMD             95%-CI
## ACT            -0.9052 [-1.1543; -0.6561]
## CBT            -1.2018 [-1.2953; -1.1084]
## Gestalt        -0.7000 [-0.9495; -0.4505]
## MBCT           -1.0664 [-1.2151; -0.9178]
## MCT            -1.1141 [-1.2309; -0.9973]
## Placebo        -0.4395 [-0.6188; -0.2602]
## Psychoanalysis -0.9439 [-1.1927; -0.6952]
## Psychodynamic  -0.8274 [-1.0401; -0.6147]
## System         -0.5700 [-0.8230; -0.3170]
## Waitlist             .                  .
## 
## Random effects model
## 
## Treatment estimate (sm = 'SMD', comparison: other treatments vs 'Waitlist'):
##                    SMD             95%-CI
## ACT            -0.7311 [-1.2918; -0.1705]
## CBT            -1.2335 [-1.4839; -0.9830]
## Gestalt        -0.7000 [-1.3927; -0.0073]
## MBCT           -1.1291 [-1.5596; -0.6986]
## MCT            -1.1268 [-1.4291; -0.8244]
## Placebo        -0.4166 [-0.8887;  0.0556]
## Psychoanalysis -0.9497 [-1.4040; -0.4955]
## Psychodynamic  -0.8418 [-1.3236; -0.3600]
## System         -0.5700 [-1.2640;  0.1240]
## Waitlist             .                  .
## 
## Quantifying heterogeneity / inconsistency:
## tau^2 = 0.1087; I^2 = 81.4%
## 
## Tests of heterogeneity (within designs) and inconsistency (between designs):
##                     Q d.f.  p-value
## Total           96.99   18 < 0.0001
## Within designs  74.46   11 < 0.0001
## Between designs 22.53    7   0.0021
netgraph(netmet,seq = c("Waitlist","Gestalt","Psychodynamic","MCT","MBCT","Placebo","CBT","System","Psychoanalysis","ACT"))

netgraph(netmet, start="circle", iterate=TRUE, col="darkgray", cex=1.5, multiarm=TRUE, points=TRUE, col.points="blue", cex.points=3)

forest(netmet, xlim=c(-1.5, 0.5), ref="Waitlist", leftlabs="Contrast to Waitlist", xlab="Effect on Depression (SMD)",sortvar = TE, smlab = "")

forest(netmet, xlim=c(-1.5, 0.5), ref="Placebo", leftlabs="Contrast to Placebo", xlab="Effect on Depression (SMD)",sortvar = TE, smlab = "")

References

Schwarzer, Guido, James R Carpenter, and Gerta Rücker. 2015. Meta-Analysis with R. Springer.

banner