12.2 Subgroup Analyses in Three-Level Models

Once our three-level meta-analytic model is set, it is easy to also check for moderators of the overall effect, i.e. conduct subgroup analyses. To do this, we can use the mods parameter in rma.mv. The subgroup/moderator can be specified by putting a tilde (~) in front of it. In our case, we want to check if dissertations report higher or lower effect sizes, so we use mods = ~ dissertation. Here’s the full code:

model.mods<-rma.mv(y,
v,
random = ~ 1 | ID_1/ID_2,
tdist = TRUE,
data = mlm.data,
method = "REML",
mods = ~ dissertation)
summary(model.mods)
##
## Multivariate Meta-Analysis Model (k = 80; method: REML)
##
##   logLik  Deviance       AIC       BIC      AICc
## -43.8184   87.6367   95.6367  105.0636   96.1847
##
## Variance Components:
##
##             estim    sqrt  nlvls  fixed     factor
## sigma^2.1  0.1594  0.3992     14     no       ID_1
## sigma^2.2  0.0581  0.2410     80     no  ID_1/ID_2
##
## Test for Residual Heterogeneity:
## QE(df = 78) = 428.5167, p-val < .0001
##
## Test of Moderators (coefficient 2):
## F(df1 = 1, df2 = 78) = 0.0226, p-val = 0.8808
##
## Model Results:
##
##               estimate      se    tval    pval    ci.lb   ci.ub
## intrcpt         0.4297  0.1229  3.4972  0.0008   0.1851  0.6744  ***
## dissertation    0.0339  0.2255  0.1504  0.8808  -0.4150  0.4828
##
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

The first important output is the Test of Moderators. We see that $$F_{1,78}=0.0226$$ with $$p=0.8808$$, meaning that there is no significant difference between subgroups. The Model Results are printed within a meta-regression framework, so we cannot simply extract the estimates directly to attain the summary effect size within subgroups. The first row, the intrcpt (intercept) is the value of $$g$$ when dissertation = 0, i.e., the value for peer-review articles. The predictor dissertation’s estimate is expressed as a slope, meaning that we have to add its value to the intercept to get the actual summary effect. This is:

$g_{dissertation} = \beta_0 + \beta_{dissertation} = 0.4297+0.0339=0.4636$