31 Tests for means of two independent groups

So far, you have learnt to ask a RQ, identify different ways of obtaining data, design the study, collect the data describe the data, summarise data graphically and numerically, understand the tools of inference, and to form confidence intervals.

In this chapter, you will learn about hypothesis tests for the difference between two means. You will learn to:

  • conduct hypothesis tests for comparing two means.
  • determine whether the conditions for using these methods apply in a given situation.

31.1 Introduction: Reaction times

A study,488 examined the reaction times of students while driving.

In one study, two different groups of students were used: one group used a mobile phone, and a different group did not use a mobile phone. Their reaction times were measured in a driving simulator. These data were seen previously in Sect. 24.1.

The two groups receive different treatments: one group used a mobile phone while driving, and a different group did not use a mobile phone while driving.

The data are not paired; instead, the means of two separate (or independent) samples are being compared. (The data would be paired if each student was measured twice: once using a phone, and once without using a phone.)

Consider the RQ:

For students, is there a difference between the mean reaction time while driving, between students who are using a mobile phone and students who are not using a mobile phone?

The data are shown below.

31.2 Hypotheses and notation: Two independent means

Since two groups are being compared, distinguishing between the statistics for the two groups (say, Group A and Group B) is important (recapping Sect. 24.3).

One way is to use subscripts (see Table 31.1). Using this notation, the parameter in the RQ is the difference between population means: \(\mu_A-\mu_B\).

As usual, the population values are unknown, so this is estimated using the statistic \(\bar{x}_A-\bar{x}_B\).

TABLE 31.1: Notation used to distinguish between the two independent groups
Group A Group B
Population means: \(\mu_A\) \(\mu_B\)
Sample means: \(\bar{x}_A\) \(\bar{x}_B\)
Standard deviations: \(s_A\) \(s_B\)
Standard errors: \(\displaystyle\text{s.e.}(\bar{x}_A) = \frac{s_A}{\sqrt{n_A}}\) \(\displaystyle\text{s.e.}(\bar{x}_B) = \frac{s_B}{\sqrt{n_B}}\)
Sample sizes: \(n_A\) \(n_B\)

For the reaction-time data, the differences are computed as the mean reaction time for phone users, minus the mean reaction time for non-phone users: \(\mu_P - \mu_C\). By this definition, the differences refer to how much greater (on average) the reaction times are when students are using phones.

The parameter is \(\mu_P - \mu_C\), the difference between the population mean reaction times (using phone, minus not using a phone).

Here the difference is computed as the mean reaction time for phone users, minus the mean reaction time for non-phone users. Computing the difference as the mean reaction time for non-phone users, minus the mean reaction time for phone users is also correct; you need to be clear about how the difference is computed, and be consistent throughout.

As always (Sect. 29.2), the null hypothesis is the default 'no difference, no change, no relationship' position; hence the null hypothesis is that there is 'no difference' between the population means of the two groups:

  • \(H_0\): \(\mu_P - \mu_C=0\) (or \(\mu_P = \mu_C\)).

This hypothesis proposes that any difference between the sample means is due to sampling variation. This becomes the initial assumption.

From the RQ, the alternative hypothesis will be two-tailed:

  • \(H_1\): \(\mu_P - \mu_C\ne 0\) (or \(\mu_P \ne \mu_C\)).

31.3 Sampling distribution: Two independent means

The data for testing the hypothesis are shown below. The numerical summary (Sect. 24.1) must summarise the difference between the means (since the RQ is about the difference), and should summarise each group.

All this information is found using software (jamovi: Fig. 31.1; SPSS: Fig. 31.2), and can be compiled into a table (Table 31.2).

The appropriate summary for graphically summarising the data is a boxplot (though a dot chart is also acceptable). An error bar chart (Fig. 24.7), which allows the sample means to be compared, should also be produced.

jamovi output for the phone reaction time data

FIGURE 31.1: jamovi output for the phone reaction time data

SPSS output for the phone reaction time data

FIGURE 31.2: SPSS output for the phone reaction time data

The difference between the sample means is 51.59 ms... but this value will vary from sample to sample; that is, there is sampling variation. The sampling variation (expectation) for the values of \(\bar{x}_A - \bar{x}_B\) can be described as having:

  • an approximate normal distribution;
  • centred around \({\mu_{P}} - {\mu_{C}} = 0\) (from \(H_0\));
  • with a standard deviation of \(\displaystyle\text{s.e.}(\bar{x}_P - \bar{x}_C)\), called the standard error for the difference between the means.

jamovi and SPSS give results from two similar hypothesis tests.

In this book, we will always use the second row of information (the "Welch's \(t\)" row in jamovi; the "Equal variance not assumed" row in SPSS) because it is more general and makes fewer assumptions.

TABLE 31.2: Numerical summaries of the reaction-time data (in milliseconds)
Mean Sample size Standard deviation Standard error
Using phone 585.19 32 89.65 15.847
Not using phone 533.59 32 65.36 11.554
All students 51.59 19.612

31.4 The test statistic: Two independent means

The observed sample mean difference (observations), relative to what was expected, is found by computing the test statistic, in this case, a \(t\)-score. The software output (jamovi: Fig. 31.1; SPSS: Fig. 31.2) can be used, but the \(t\)-score can also be computed manually:

\[\begin{align*} t &= \frac{\text{sample statistic} - \text{assumed population parameter, from $H_0$}} {\text{standard error for sample statistic}}\\ &= \frac{ (\bar{x}_P - \bar{x}_C) - (\mu_P - \mu_C)} {\text{s.e.}(\bar{x}_P - \bar{x}_C)}\\ &= \frac{51.594 - 0}{19.612} = 2.631, \end{align*}\] as in the software output.

31.5 \(P\)-values: Two independent means

A \(P\)-value is needed to determine if the sample statistic is consistent with the assumption. Since the \(t\)-score is large, the \(P\)-value will be small using the 68--95--99.7 rule.

This is confirmed by the software (jamovi: Fig. 31.1; SPSS: Fig. 31.2): the two-tailed \(P\)-value is \(0.011\). The small \(P\)-value suggests that the observations are inconsistent with the assumption (Table 29.1).

Click on the hotspots in the following image, to see what the SPSS output tells us.

31.6 Conclusions: Two independent means

In conclusion we write:

Moderate evidence exists in the sample (two independent samples \(t=2.631\); two-tailed \(P=0.011\)) that the population mean reaction time is different for students using a mobile phone (mean: 585.19 ms; \(n=32\)) and students not using a mobile phone (mean: 533.59ms; \(n = 32\); 95% CI for the difference: \(12.4\) to \(90.9\)ms longer for phone users).

Again, the conclusions contains an answer to the RQ, the evidence leading to this conclusion (\(t=2.631\); two-tailed \(P=0.011\)), and some sample summary statistics, including a CI.

31.7 Statistical validity conditions: Two independent means

As usual, these results apply under certain conditions, which are the same as those for forming a CI for the difference between two means.

The test above is statistically valid if one of these conditions is true:

  1. Both sample sizes are at least 25; or
  2. Either sample size is smaller than 25, and both populations have an approximate normal distribution.

The sample size of 25 is a rough figure here, and some books give other values (such as 30). We can explore the histograms of the samples to determine if normality of the populations seems reasonable.

In addition to the statistical validity condition, the test will be

Example 31.1 (Statistical validity) For the reaction-time data, both samples sizes are \(n=32\). This means that the results will be statistically valid.

Explicitly, the data in each group do not need be normally distributed, since both sample sizes are larger than 25.

Example 31.2 (Gray whales) A study of gray whales (Eschrichtius robustus) measured (among other things) the length of adult whales.489 The data are shown below.

Sex Mean (in m) Standard deviation (in m) Sample size
Female 12.70 0.611 260
Male 12.07 0.705 139

Are adult female gray whales longer than males, on average?

Let's define the difference as the mean length of female gray whales minus the mean length of male gray whales. Then we wish to estimate the difference \(\mu_F - \mu_M\), where \(F\) and \(M\) represent female and male gray whales respectively; this is the parameter of interest. The best estimate of this difference is \(\bar{x}_F - \bar{x}_M = 12.70 - 12.07 = 0.63\) m.

The hypotheses are:

  • \(H_0\): \(\mu_F - \mu_M = 0\)
  • \(H_1\): \(\mu_F - \mu_M \ne 0\)

We know that the difference between the sample means is likely to vary from sample to sample, and hence it has a standard error.

We cannot easily determine the standard error of this difference from the above information (though it is possible), so we must be given this information: \(\text{s.e.}(\bar{x}_F - \bar{x}_M) = 0.07079\).

The test statistic is

\[ t = \frac{(\bar{x}_F - \bar{x}_M) - (\mu_F - \mu_M)}{\text{s.e.}(\bar{x}_F - \bar{x}_M)} = \frac{0.63 - 0}{0.07079} = 8.90, \] which is very large. This means that the \(P\)-value will be very small (using the 68-95--99.7 rule).

We write:

There is very strong evidence (\(t = 8.90\); two-tailed \(P < 0.001\)) that the mean length of adult gray whales is different for females (mean: 12.70 m; standard deviation: 0.611 m) and males (mean: 12.07 m; standard deviation: 0.705 m; 95% CI for the difference: 0.48 m to 0.77 m).

Since both sample sizes are large, the test is statistically valid.

(Check that you can compute the correct CI!)

31.8 Example: Speed signage

In an attempt to reduce vehicle speeds on freeway exit ramps, a Chinese study tried using additional signage.490

At one site studied (Ningxuan Freeway), speeds were recorded for 38 vehicles before the extra signage was added, and then for 41 vehicles after the extra signage was added.

The researchers are hoping that the addition of extra signage will reduce the mean speed of the vehicles.

The RQ is:

At this freeway exit,did the mean vehicle speed reduce after extra signage was added?

The data are not paired: different vehicles are measured before and after the extra signage is added.

The data are summarised in Table 31.3.

TABLE 31.3: The signage data summary (in km/h)
Mean Std deviation Std error Sample size
Before 98.02 13.19 2.140 38
After 92.34 13.13 2.051 41
Speed reduction 5.68 0.472

The standard error must given; you cannot easily calculate this from the other information. You are not expected to do so.

A useful graphical summary of the data is a boxplot (Fig. 24.12, left panel); likewise, an error bar chart can be produced by computing the CI for each group (Fig. 24.12, right panel).

The parameter is \(\mu_{\text{Before}} - \mu_{\text{After}}\), the reduction in the mean speed.

The hypotheses are:

  • \(H_0\): \(\mu_{\text{Before}} - \mu_{\text{After}} = 0\): There is no change in the mean speeds
  • \(H_1\): \(\mu_{\text{Before}} - \mu_{\text{After}} > 0\): The mean speed has reduced

The best estimate of the difference in population means is the difference between the sample means: \((\mu_{\text{Before}} - \mu_{\text{After}}) = 5.68\). The table also gives the standard error for estimating this difference as \(\text{s.e.}(\bar{x}_{\text{Before}} - \bar{x}_{\text{After}}) = 0.472\).

Using the summary information in Table 31.3, the \(t\)-score is computed using Equation (28.1):

\[ t = \frac{5.68 - 0}{2.96} = 1.92. \] (Recall that \(\mu_{\text{Before}} - \mu_{\text{After}} = 0\) from the null hypothesis.)

Using the 68--95--99.7 rule, this \(t\)-score is not very small.

Remembering that the alternative hypothesis is one-tailed, the \(P\)-value is larger than \(0.025\), but smaller than \(0.32\)... so it is difficult to make a definite decision without using software. However, since the \(t\)-score is just less than 2, we suspect that the \(P\)-value is likely to be closer to \(0.025\) than \(0.32\).

In fact, from software we find that \(P = 0.02968\) (you cannot be this precise just using the 68--95--99.7 rule). Using Table 29.1, this \(P\)-value only provides moderate evidence of a reduction in mean speeds.

We conclude:

Moderate evidence exists in the sample (\(t = 1.92\); two-tailed \(P = 0.030\))
that mean speeds have reduced after the addition of extra signage (Before: mean speed: 98,02 km/h; \(n = 38\); standard deviation: \(13.18\) km/h; After: mean speed: 92.34 km/h; \(n = 41\); standard deviation: \(13.13\) km/h; 95% CI for the difference: -0.24 to 11.6 km/h).

Using the validity conditions, the CI is statistically valid.

Remember: clearly state which mean is larger.

31.9 Example: Health Promotion services

A study491 compared the access to health promotion (HP) services for people with and without a disability. (This study was seen in Sect. 24.11.) Access was measured using the Barriers to Health Promoting Activities for Disabled Persons BHADP scale, where higher scores mean greater barriers. The RQ is:

Is the mean BHADP score the same for people with and without a disability?

The parameter is \(\mu_D - \mu_N\), the difference between the population mean BHADP score (people with disabilities, minus people without disabilities).

In this case, only numerical summary data is available (Table 31.4), not the original data. (An appropriate graphical summary, an error bar chart, can be constructed from the summary information (Fig. 24.13, though a boxplot cannot be constructed from the information.) Denoting those with and without a disability with subscripts \(D\) and \(N\) respectively, the hypotheses are:

  • \(H_0\): \(\mu_D - \mu_N = 0\): There is no difference in the population mean BHADP scores
  • \(H_1\): \(\mu_D - \mu_N \ne 0\): There is a difference in the population mean BHADP scores
TABLE 31.4: The BHADP data summary
Sample mean Std deviation Sample size Std error
Disability 31.83 7.73 132 0.6728
No disability 25.07 4.8 137 0.4101
Difference 6.76 0.80285

The best estimate of the difference in population means is the difference between the sample means: \((\bar{x}_D - \bar{x}_{ND}) = 6.76\). The table also gives the standard error for estimating this difference as \(\text{s.e.}(\bar{x}_D - \bar{x}_{ND}) = 0.80285\) (as given in the article).

The standard error is given here; you cannot easily calculate this from the given information. You are not expected to do so.

Using the summary information in Table 31.4, the \(t\)-score is computed using Equation (28.1):

\[ t = \frac{6.76 - 0}{0.80285} = 8.42. \] (Recall that \(\mu_D - \mu_N = 0\) from the null hypothesis.) Using the 68--95--99.7 rule, this very large \(t\)-score implies the \(P\)-value will be very small. We conclude:

Strong evidence exists in the sample (\(t = 8.42\); two-tailed \(P < 0.001\)) that people with a disability (mean: 31.83; \(n = 132\); standard deviation: \(7.73\)) and people without a disability (mean: 25.07; \(n = 137\); standard deviation: \(4.80\)) have different population mean access to health promotion services (95% CI for the difference: 5.17 to 8.35).

31.10 Example: Face-plant study

A study492 compared the lean-forward angle in younger and older women. (This study was seen in Sect. 24.12.) An elaborate set-up was constructed to measure this lean-forward angle, using harnesses.

Consider this RQ:

Among healthy women, is the mean lean-forward angle greater for younger women compared to older women?

The parameter is \(\mu_Y - \mu_O\), the difference between the population mean lean-forward angle (younger women, minus older women).

This is a one-tailed RQ. Denoting the younger and older women with subscrpts \(Y\) and \(O\) respectively, the hypotheses are:

  • \(H_0\): \(\mu_{\text{Y}} - \mu_{\text{O}} = 0\) (or \(\mu_{\text{Y}} = \mu_{\text{O}}\)): There is no difference in the population mean lean-forward angle between the two age groups (the assumption);
  • \(H_1\): \(\mu_{\text{Y}} - \mu_{\text{O}} > 0\) (or \(\mu_{\text{Y}} > \mu_{\text{O}}\)): There is a difference in the population mean lean-forward angle between the two groups.

The data (Table 24.5), numerical summary (Table 24.6). and error bar chart (Fig. 24.14) were shown in Sect. 24.12.

Using the sampling distribution (expectation), the \(t\)-score can be found on the the software output (jamovi: Fig. 31.3; SPSS: Fig. 31.3), or manually:

\[ t = \frac{14.5-0}{2.167} = 6.691 \] (observation). The two-tailed \(P\)-value is \(0.001\), so the one-tailed \(P\)-value is \(0.001\div 2 = 0.0005\). This is very small, so we conclude:

Very strong evidence exists in the sample (\(t = 6.691\); one-tailed \(P = 0.0005\)) that the population mean one-step fall recovery angle for healthy women is greater for young women (mean: \(30.7^\circ\); std. dev.: \(2.58^\circ\); \(n = 10\)) compared to older women (mean: \(16.20^\circ\); std. dev.: \(4.44^\circ\); \(n = 5\); 95% CI for the difference: \(9.1^\circ\) to \(19.9^\circ\)).

jamovi output for the face-plant data

FIGURE 31.3: jamovi output for the face-plant data

SPSS output for the face-plant data

FIGURE 31.4: SPSS output for the face-plant data

The sample sizes are both small, so the test may not be statistical valid. However, since the \(P\)-value is so small, the conclusion is unlikely to change substantially.

31.11 Summary

To test a hypothesis about a difference between two population means \(\mu_1 - \mu_2\), based on the value of the difference between two sample mean \(\bar{x}_1 - \bar{x}_2\), assume the value of \(\mu_1 - \mu_2\) in the null hypothesis to be true (usually zero). Then, the difference between the sample means varies from sample to sample and, under certain statistical validity conditions, varies with an approximate normal distribution centered around the hypothesised value of \(\mu_1 - \mu_2\), with a standard deviation of \(\text{s.e.}(\bar{x}_1 - \bar{x}_2)\). This distribution describes what values of the sample mean could be expected in the sample if the value of \(\mu\) in the null hypothesis was true. The test statistic is

\[ t = \frac{ (\bar{x}_1 - \bar{x}_2) - (\mu_1 - \mu_2)}{\text{s.e.}(\bar{x}_1 - \bar{x}_2)}, \] where \(\mu_1 - \mu_2\) is the hypothesised value given in the null hypothesis. This describes the observations. The \(t\)-value is like a \(z\)-score, and so an approximate \(P\)-value is estimated using the 68--95--99.7 rule, which is how we weigh the evidence to determine if it is consistent with the assumption.

The following short video may help explain some of these concepts:

31.12 Quick review questions

A study493 compared using a vegan (\(n = 46\)) and a conventional (\(n = 47\)) diet for 12 weeks, for a group of Koreans with Type II diabetes. A summary of the data for iron levels are shown in Table 31.5.

TABLE 31.5: Comparing the iron levels (mg) for subjects using a vegan or conventional diet for 12 weeks
Mean Std. dev n
Vegan diet 13.9 2.3 46
Conventional diet 15 2.7 47
Difference 1.1
  1. The sample size is missing from the 'Difference' row. What should the sample size in this row be?

  2. What is the standard deviation for the difference?

  3. What is the standard error for the difference?

  4. The two-tailed \(P\)-value for the comparison is given as \(P=0.046\). What does this mean?

Progress:

31.13 Exercises

Selected answers are available in Sect. D.29.

Exercise 31.1 Earlier, the NHANES study (Sect. 12.9; Exercise 24.1), was used to address this RQ:

Among Americans, is the mean direct HDL cholesterol different for current smokers and non-smokers?

Use the SPSS output in Fig. 31.5 to perform a hypothesis test to answer the RQ.

SPSS output for the NHANES data

FIGURE 31.5: SPSS output for the NHANES data

Exercise 31.2 A study of male paramedics in Western Australia compared conventional paramedics with special operations paramedics.494 Some information comparing their physical profiles is shown in Table 31.6.

  1. Compute the missing standard errors.
  2. Consider comparing the mean grip strength for the two groups of paramedics. The standard error for the difference between the means is 3.3044.
    • Carefully write down the hypotheses.
    • Compute the \(t\)-score for testing if a difference exists between the two types of paramedics.
    • Approximate the \(P\)-value using the 68--95--99.7 rule.
    • Discuss the conditions required for statistical validity in this context.
    • Make a conclusion.
  3. Consider comparing the mean number of push-ups completed in one minute. The standard error for the difference between the means is 4.0689.
    • Carefully write down the hypotheses.
    • Compute the \(t\)-score for testing if a difference exists between the two types of paramedics.
    • Approximate the \(P\)-value using the 68--95--99.7 rule.
    • Discuss the conditions required for statistical validity in this context.
    • Make a conclusion.
TABLE 31.6: The physical profile of conventional (\(n=18\)) and special operation (\(n=11\)) paramedics in Western Australia
Conventional Special Operations
Grip strength (in kg)
Mean 51 56
Std deviation 8 9
Std error
Push-ups (per minutes)
Mean 36 47
Std deviation 10 11
Std error

Exercise 31.3 Consider again the body temperature data from Sect. 28.1. The researchers also recorded the gender of the patients, as they also wanted to compare the mean internal body temperatures for males and females.

Use the jamovi output in Fig. 31.6 to perform this test and make a conclusion. Also comment on the practical significance of your results.

jamovi output for the body-temperature data

FIGURE 31.6: jamovi output for the body-temperature data

Exercise 31.4 A study495 examined the sugar consumption in industrialised (mean: 41.8 kg/person/year) and non-industrialised (mean: 24.6 kg/person/year) countries.

The jamovi output is shown in Fig. 31.7.

  1. Write the hypotheses.
  2. Write down and interpret the CI.
  3. Write a conclusion for the hypothesis test.
jamovi output for the sugar-consumption data

FIGURE 31.7: jamovi output for the sugar-consumption data

In the image below, click on the location where the correct \(P\)-value is located.