Chapter 2 Example sample size calculations

We will now see some examples of sample size calculations for the following tests:

One-sample \(t\)-test
Independent samples \(t\)-test
Paired \(t\)-test
Two-sample test of proportions

To carry out sample size calculations, we need to make some assumptions, or educated guesses, about other factors such as the standard deviation and effect size. Pilot studies are very helpful in this regard: they involve carrying out the study but on a much smaller scale, in preparation for the main study. If a pilot study is carried out first, then this can help to determine the estimated standard deviation and other factors required for the sample size calculation.

While the results of the example sample size calculations are provided here, in this topic's computer lab we will learn how to carry out sample size calculations using a software package called G*Power (Faul et al. 2007, 2009).

2.0.1 One-sample \(t\)-test

Suppose we wish to design a study to determine whether the average cholesterol level of patients from a particular population is different from 5.0 mmol/L. Further suppose the following:

The hypothesis test to be carried out will be a two-tailed test
The significance level is \(\alpha = 0.05\)
For the purposes of this study, a mean difference of at least 0.5 mmol/L is considered meaningful
Based on a pilot study, the estimated standard deviation for this population is 0.55
We wish to choose a sample size to ensure a power of at least 0.8.

Then, it turns out that the required sample size would be \(n = 12\).

Your turn

Suppose now that instead of aiming for a power of 0.8, we are aiming for a power of 0.9. One of the options in the drop-down menu would be the required sample size (assuming all of the other parameters outlined above remain the same) to achieve a power of 0.9. Which one must it be?

2.0.2 Independent samples \(t\)-test

Suppose we wish to design a study to determine whether the average cholesterol level is significantly different between two different groups of patients (Group A and Group B). Further suppose the following:

The hypothesis test to be carried out will be a two-tailed test
The significance level is \(\alpha = 0.05\)
Based on a pilot study, the estimated mean cholesterol level for Group A is 5.0 mmol/L
For the purposes of this study, a difference in mean of at least 0.3 between groups would be considered meaningful
Based on a pilot study, the estimated standard deviation for both groups is 0.55
We wish to choose a sample size to ensure a power of at least 0.8.

Then, it turns out that the required sample size for each group would be \(n_1 = n_2 = 54\) for a total sample size of \(n = 108\).

2.0.3 Paired \(t\)-test

Suppose we wish to design a study to determine whether the average difference in before and after weights of a cohort of anorexia patients was statistically significant. Further suppose the following:

The hypothesis test to be carried out will be a two-tailed test
The significance level is \(\alpha = 0.05\)
Based on a pilot study, the estimated mean starting weight is 36.6 kg
For the purposes of this study, an average change between before and after weights of at least 1.1kg would be considered meaningful
Based on a pilot study, the estimated standard deviations for the before and after weights are 2.1 and 3.5 respectively
Based on a pilot study, the correlation between the matched pair before and after weights is estimated to be 0.63
We wish to choose a sample size to ensure a power of at least 0.8.

Then, the required sample size would be \(n = 50\) participants.

2.0.4 Two-sample test of proportions

Suppose we wish to design a study to determine whether there is a significant difference in the proportion of US adults who say they use Facebook between two groups: those aged 18-29, and those aged 30-49. Further suppose the following:

The hypothesis test to be carried out will be a two-tailed test
The significance level is \(\alpha = 0.05\)
Based on a pilot study, the estimated percentage of those aged 18-29 who say they use Facebook is 70%
For the purposes of this study, an observed difference between the two age groups of at least 10 percentage points would be considered meaningful
We wish to choose a sample size to ensure a power of at least 0.8.

Then, the required sample size for each group would be \(n_1 = n_2 = 294\) for a total sample size of \(n = 588\).

References

Faul, Franz, Edgar Erdfelder, Axel Buchner, and Albert-Georg Lang. 2009. “Statistical Power Analyses Using g* Power 3.1: Tests for Correlation and Regression Analyses.” Behavior Research Methods 41 (4): 1149–60.

Faul, Franz, Edgar Erdfelder, Albert-Georg Lang, and Axel Buchner. 2007. “G* Power 3: A Flexible Statistical Power Analysis Program for the Social, Behavioral, and Biomedical Sciences.” Behavior Research Methods 39 (2): 175–91.