16.2 Random-Effects Model
For power analyses under the random-effects model, the formula to calculate the variance of my true mean effect looks slightly different:
We see that again, \(tau^2\) has to be included to take the between-study heterogeneity into account (see Chapter 4.2 for more details). However, i do not know the between-study heterogeneity of my analysis before i perform it, so what value should i assume?
According to Hedges and Pigott (Hedges and Pigott 2004), the follwing formulae may be used to calculate the power in the random-effect model assuming small, moderate or high heterogeneity:
Again, you don’t have to worry about the statistical details here. We have put the entire calculations into the
power.analysis function, which can already introduced before.
I will assume the same parameters used for the fixed-effect model power analysis, but this time i will also have to specify the
heterogeneity in the function, which can take the values
"high". I will choose
"moderate" for this example.
power.analysis(d=0.30, k=10, n1=25, n2=25, p=0.05, heterogeneity = "moderate")
The output i get is:
## Random-effects model used (moderate heterogeneity assumed).
##  0.7327163
Interestingly, we see that this value is 73%, which is smaller than the value of 91% which was calculated using the fixed-effect model. The value is also below 80%, meaning that i would not have optimal power to find the desired effect of \(d=0.30\) to be statistically significant if it exists.
This has to do with the larger heterogeneity i assume in this simulation, which decreases the precision of my effect size estimate, and thus increases my need for statistical power.
The graph below visualizes this relationship:
Hedges, Larry V, and Therese D Pigott. 2004. “The Power of Statistical Tests for Moderators in Meta-Analysis.” Psychological Methods 9 (4). American Psychological Association: 426.