32 Tests for means of two independent groups

So far, you have learnt to ask a RQ, design a study, describe and summarise the data, understand the decision-making process and to work with probabilities. You have been introduced to the construction of confidence intervals, and have studied some hypothesis tests. In this chapter, you will learn about hypothesis tests for the difference between two means. You will learn to:

conduct hypothesis tests for comparing two means.
determine whether the conditions for using these methods apply in a given situation.

32.1 Introduction: reaction times

To study the reaction times of students while driving, two different groups of students were used: one group used a mobile phone, and a different group did not use a mobile phone. Their reaction times were measured in a driving simulator (Strayer and Johnston 2001). These data were seen previously in Sect. 24.1.

The two groups of students receive different treatments: one group used a mobile phone while driving, and a different group did not use a mobile phone while driving. This is an example of a between-individuals comparison. The data are not paired; instead, the means of two separate (or independent) samples are being compared. The data would be paired if each student was measured twice: once using a phone, and once without using a phone.

Consider the RQ:

For students, is the mean reaction time while driving the same for students who are using a mobile phone and students who are not using a mobile phone?

The data are shown below.

32.2 Hypotheses and notation

Since two groups are being compared, using subscripts to distinguish between the statistics for the two groups (in general, Group A and Group B) is important (recapping Sect. 24.2); see Table 32.1. Using this notation, the parameter in the RQ is the difference between population means: $\mu_A - \mu_B$. As usual, the population values are unknown, so this is estimated using the statistic $\bar{x}_A - \bar{x}_B$.

TABLE 32.1: Notation used to distinguish between the two independent groups
	Group A	Group B
Population means:	$\mu_A$	$\mu_B$
Sample means:	$\bar{x}_A$	$\bar{x}_B$
Standard deviations:	$s_A$	$s_B$
Standard errors:	$\displaystyle\text{s.e.}(\bar{x}_A) = \frac{s_A}{\sqrt{n_A}}$	$\displaystyle\text{s.e.}(\bar{x}_B) = \frac{s_B}{\sqrt{n_B}}$
Sample sizes:	$n_A$	$n_B$

For these data, the differences are computed as the mean reaction time for phone users ($P$), minus the mean reaction time for non-phone users ($C$; the control group): $\mu_P - \mu_C$. By this definition, the differences refer to how much greater (on average) the reaction times are when students are using phones. The parameter is $\mu_P - \mu_C$, the difference between the population mean reaction times (using phone, minus not using a phone).

Here the difference is computed as the mean reaction time for phone users, minus the mean reaction time for non-phone users. Computing the difference as the mean reaction time for non-phone users, minus the mean reaction time for phone users is also correct. You need to be clear about how the difference is computed, and be consistent throughout.

As always (Sect. 30.2), the null hypothesis is the default 'no difference, no change, no relationship' position; any difference is due to sampling variation. Hence, the null hypothesis is that there is 'no difference' between the population means of the two groups:

$H_0$: $\mu_P - \mu_C = 0$ (or $\mu_P = \mu_C$).

This hypothesis proposes that any difference between the sample means is due to sampling variation, and is the initial assumption. From the RQ, the alternative hypothesis will be two-tailed:

$H_1$: $\mu_P - \mu_C\ne 0$ (or $\mu_P \ne \mu_C$).

This hypothesis proposes that any difference between the sample means is because a difference really exists between the population means.

32.3 Describing the sampling distribution

The data for testing the hypothesis are shown below. The numerical summary (Sect. 24.1) must summarise the difference between the means (since the RQ is about the difference), and should summarise each group.

All this information is found using software (jamovi: Fig. 32.1; SPSS: Fig. 32.2), and can be compiled into a table (Table 32.2). The appropriate summary for graphically summarising the data is a boxplot (though a dot chart is also acceptable). An error bar chart (Fig. 24.5), which compares the sample means, should also be produced.

FIGURE 32.1: jamovi output for the phone reaction time data

FIGURE 32.2: SPSS output for the phone reaction time data

The difference between the sample means is 51.59 ms... but this value will vary from sample to sample; that is, there is sampling variation. The sampling variation (expectation) for the values of $\bar{x}_A - \bar{x}_B$ can be described as having:

an approximate normal distribution;
centred around a sampling mean whose value is ${\mu_{P}} - {\mu_{C}} = 0$, the difference between the means (from $H_0$);
with a standard deviation of $\displaystyle\text{s.e.}(\bar{x}_P - \bar{x}_C)$, called the standard error for the difference between the means.

jamovi and SPSS give results from two similar hypothesis tests. In this book, we will always use the second row of information: the "Welch's $t$" row in jamovi, and the "Equal variance not assumed" row in SPSS (these two are equivalent).

In both cases, this row of information does not assume that the population standard deviation are equal. The top row of information does assume the population standard deviation are equal. In most cases, the information in both rows are similar anyway.

TABLE 32.2: Numerical summaries of the reaction-time data (in milliseconds)
	Mean	Sample size	Standard deviation	Standard error
Using phone	533.59	32	65.36	11.554
Not using phone	585.19	32	89.65	15.847
All students	-51.59			19.612

32.4 Computing the test statistic

The observed difference between sample means (observations), relative to what was expected, is found by computing the test statistic; in this case, a $t$-score. The software output (jamovi: Fig. 32.1; SPSS: Fig. 32.2) can be used, but the $t$-score can also be computed manually:
\[\begin{align*} t &= \frac{\text{sample statistic} - \text{assumed population parameter, from $H_0$}} {\text{standard error for sample statistic}}\\ &= \frac{ (\bar{x}_P - \bar{x}_C) - (\mu_P - \mu_C)} {\text{s.e.}(\bar{x}_P - \bar{x}_C)}\\ &= \frac{51.594 - 0}{19.612} = 2.631, \end{align*}\] as in the software output.

32.5 Determining $P$-values

A $P$-value determines if the sample statistic is consistent with the assumption. Since the $t$-score is large, the $P$-value will be small using the 68--95--99.7 rule.

This is confirmed by the software (jamovi: Fig. 32.1; SPSS: Fig. 32.2): the two-tailed $P$-value is $0.011$. The small $P$-value suggests the observations are inconsistent with the assumption (Table 30.1), and the difference between the sample means could be reasonably explained by sampling variation.

Click on the pins in the following image, and describe what the jamovi output tells us.

32.6 Writing conclusions

In conclusion, write:

Moderate evidence exists in the sample (two independent samples $t = 2.631$; two-tailed $P = 0.011$) that the population mean reaction time is different for students using a mobile phone (mean: 585.19 ms; $n = 32$) and students not using a mobile phone (mean: 533.59ms; $n = 32$; 95% CI for the difference: $12.4$ to $90.9$ms longer for phone users).

Again, the conclusions contains an answer to the RQ, the evidence leading to this conclusion ($t = 2.631$; two-tailed $P = 0.011$), and some sample summary statistics, including a CI.

32.7 Statistical validity conditions

As usual, these results apply under certain conditions, which are the same as those for forming a CI for the difference between two means. The test above is statistically valid if one of these conditions is true:

Both sample sizes are at least 25; or
Either sample size is smaller than 25, and both populations have an approximate normal distribution.

The sample size of 25 is a rough figure here, and some books give other values (such as 30). This condition ensures that the distribution of the difference between sample means has an approximate normal distribution (so that, for example, the 68--95--99.7 rule can be used). We can explore the histograms of the samples to determine if normality of the populations seems reasonable.

Example 32.1 (Statistical validity) For the reaction-time data, both samples sizes are $n = 32$, so the results will be statistically valid. The data in each group do not need be normally distributed, since both sample sizes are larger than 25.

32.8 Example: speed signage

In an attempt to reduce vehicle speeds on freeway exit ramps, a Chinese study added signage (Ma et al. 2019). At one site studied (Ningxuan Freeway), speeds were recorded for 38 vehicles before the extra signage was added, and then for 41 vehicles after the extra signage was added (data with Sect. 24.9).

The researchers are hoping the addition of extra signage will reduce the mean speed of the vehicles. The RQ is:

At this freeway exit, does the mean vehicle speed reduce after extra signage is added?

The data are not paired: different vehicles are measured before and after the extra signage is added. The data are summarised in Table 32.3.

TABLE 32.3: The signage data summary (in km/h)
	Mean	Std deviation	Std error	Sample size
Before	98.02	13.19	2.140	38
After	92.34	13.13	2.051	41
Speed reduction	5.68		0.472

You cannot easily calculate the standard error of the difference (i.e., $\text{s.e.}(\bar{x}_{\text{Before}} - \bar{x}_{\text{After}})$) from the other information. You are not expected to do so; the standard error must given (e.g., from software output).

A graphical summary of the data is a boxplot (Fig. 24.8, left panel); an error bar chart compares the means, with a CI for each group (Fig. 24.8, right panel).

The parameter is $\mu_{\text{Before}} - \mu_{\text{After}}$, the reduction in the populations mean speed after signage is added. The hypotheses are:

$H_0$: $\mu_{\text{Before}} - \mu_{\text{After}} = 0$: There is no change in the population mean speeds.
$H_1$: $\mu_{\text{Before}} - \mu_{\text{After}} > 0$: The population mean speed has reduced.

The best estimate of the difference in population means is the difference between the sample means: $(\bar{x}_{\text{Before}} - \bar{x}_{\text{After}}) = 5.68$. The table gives the standard error for estimating this difference as $\text{s.e.}(\bar{x}_{\text{Before}} - \bar{x}_{\text{After}}) = 0.472$. Using the summary information in Table 32.3, the $t$-score (using Equation (29.1)) is
\[ t = \frac{(\bar{x}_B - \bar{x}_A) - (\mu_B - \mu_A)}{\text{s.e.}(\bar{x}_B - \bar{x}_{A})} = \frac{5.68 - 0}{2.96} = 1.92. \] (Recall that $\mu_{\text{Before}} - \mu_{\text{After}} = 0$ from the null hypothesis.)

Remembering that the alternative hypothesis is one-tailed, the $P$-value (using the 68--95--99.7 rule) is larger than $0.025$, but smaller than $0.32$... so making a clear decision is difficult without using software. However, since the $t$-score is just less than 2, we suspect that the $P$-value is likely to be closer to $0.025$ than $0.32$.

From software, $P = 0.02968$ (you cannot be this precise just using the 68--95--99.7 rule). Using Table 30.1, this $P$-value only provides moderate evidence of a reduction in mean speeds. We conclude:

Moderate evidence exists in the sample ($t = 1.92$; one-tailed $P = 0.030$) that mean speeds have reduced after the addition of extra signage (Before: mean speed: $98.02$ km/h; $n = 38$; standard deviation: $13.18$ km/h; After: mean speed: $92.34$ km/h; $n = 41$; standard deviation: $13.13$ km/h; 95% CI for the difference: -0.24 to 11.6 km/h).

Whether the mean speed reduction of 5.68 has practical importance is a different issue. Using the validity conditions, the CI is statistically valid.

Remember: which mean is larger must be clear!

32.9 Example: health promotion services

A study (Becker, Stuifbergen, and Sands 1991) compared access to health promotion (HP) services for people with and without a disability. (This study was seen in Sect. 24.10.) Access was measured using the Barriers to Health Promoting Activities for Disabled Persons BHADP scale, where higher scores mean greater barriers. The RQ is:

Is the mean BHADP score the same for people with and without a disability?

The parameter is $\mu_D - \mu_N$, the difference between the population mean BHADP score (people with disabilities, minus people without disabilities).

In this case, only numerical summary data are available (Table 32.4), not the original data. (An appropriate graphical summary, an error bar chart, can be constructed from the summary information (Fig. 24.9), though a boxplot cannot be constructed from the information.) Denoting those with and without a disability with subscripts $D$ and $N$ respectively, the hypotheses are:

$H_0$: $\mu_D - \mu_N = 0$: There is no difference in the population mean BHADP scores.
$H_1$: $\mu_D - \mu_N \ne 0$: There is a difference in the population mean BHADP scores.

TABLE 32.4: The BHADP data summary
	Sample mean	Std deviation	Sample size	Std error
Disability	31.83	7.73	132	0.6728
No disability	25.07	4.8	137	0.4101
Difference	6.76			0.80285

The best estimate of the difference between the population means is the difference between the sample means: $(\bar{x}_D - \bar{x}_{ND}) = 6.76$. The table also gives the standard error for estimating this difference as $\text{s.e.}(\bar{x}_D - \bar{x}_{ND}) = 0.80285$ (given in the article), since the difference will vary for all possible samples.

The standard error is given; you cannot easily calculate this from the given information. You are not expected to do so.

Using the summary information in Table 32.4, the $t$-score is computed using Equation (29.1):

\[ t = \frac{(\bar{x}_D - \bar{x}_{ND}) - (\mu_D - \mu_{ND})}{\text{s.e.}(\bar{x}_D - \bar{x}_{ND})} = \frac{6.76 - 0}{0.80285} = 8.42. \] (Recall that $\mu_D - \mu_N = 0$ from the null hypothesis.) Using the 68--95--99.7 rule, this very large $t$-score implies the $P$-value will be very small. We conclude:

Strong evidence exists in the sample ($t = 8.42$; two-tailed $P < 0.001$) that people with a disability (mean: 31.83; $n = 132$; standard deviation: $7.73$) and people without a disability (mean: 25.07; $n = 137$; standard deviation: $4.80$) have different population mean access to health promotion services (95% CI for the difference: 5.17 to 8.35), as measured by BHADP scores.

32.10 Summary

To test a hypothesis about a difference between two population means $\mu_1 - \mu_2$, based on the value of the difference between two sample mean $\bar{x}_1 - \bar{x}_2$, assume the value of $\mu_1 - \mu_2$ in the null hypothesis to be true (usually zero). Then, the difference between the sample means varies from sample to sample and, under certain statistical validity conditions, varies with an approximate normal distribution centred around the hypothesised value of $\mu_1 - \mu_2$, with a standard deviation of $\text{s.e.}(\bar{x}_1 - \bar{x}_2)$. This distribution describes what values of the sample mean could be expected in the sample if the value of $\mu$ in the null hypothesis was true. The test statistic is

\[ t = \frac{ (\bar{x}_1 - \bar{x}_2) - (\mu_1 - \mu_2)}{\text{s.e.}(\bar{x}_1 - \bar{x}_2)}, \] where $\mu_1 - \mu_2$ is the hypothesised value given in the null hypothesis (usually zero). This describes the observations. The $t$-value is like a $z$-score, and so an approximate $P$-value is estimated using the 68--95--99.7 rule or normal-distribution tables, which is how we weigh the evidence to determine if it is consistent with the assumption.

The following short video may help explain some of these concepts:

32.11 Quick review questions

A study (Y.-M. Lee et al. 2016) compared using a vegan ($n = 46$) and a conventional ($n = 47$) diet for 12 weeks, for a group of Koreans with Type II diabetes. A summary of the data for iron levels are shown in Table 32.5.

TABLE 32.5: Comparing the iron levels (mg) for subjects using a vegan or conventional diet for 12 weeks
	Mean	Standard deviation	n
Vegan diet	13.9	2.3	46
Conventional diet	15.0	2.7	47
Difference	1.1

The sample size is missing from the 'Difference' row. What should the sample size in this row be?

46 + 47 = 9346 - 47 = -147 - 46 = 1A sample size makes no senseThere is not enough information
What is the standard deviation for the difference?
What is the standard error for the difference?
The two-tailed $P$-value for the comparison is given as $P = 0.046$. What does this mean?

There is no evidence that the sample means are differentThere is some evidence that the sample means are differentThere is no evidence that the population means are differentThere is some evidence that the population means are differentThere is not enough information

32.12 Exercises

Selected answers are available in Sect. D.30.

Exercise 32.1 (These data were also seen in Exercise 24.1.) A study of gray whales (Eschrichtius robustus) measured (among other things) the length of adult whales (Agbayani, Fortune, and Trites 2020). Are adult female gray whales longer than males, on average, in the population? Summary information is shown in Table 32.6. In addition, the standard error of this difference is $\text{s.e.}(\bar{x}_F - \bar{x}_M) = 0.0929$.

TABLE 32.6: Numerical summary of length of whales at birth (in m)
	Mean	Std deviation	Sample size
Female	4.66	0.379	26
Male	4.60	0.305	30

Define the difference.
Write the hypotheses to answer the RQ.
Compute the $t$-score, and approximate the $P$-value using the 68--95--99.7 rule.
Write a conclusion.
Is the test likely to be statistically valid?

Exercise 32.2 Earlier, the NHANES study (Sect. 12.10; Exercise 24.2) was used to answer this RQ:

Among Americans, is the mean direct HDL cholesterol different for current smokers and non-smokers?

Use the SPSS output in Fig. 32.3 to perform a hypothesis test to answer the RQ.

FIGURE 32.3: SPSS output for the NHANES data

Exercise 32.3 A study of male paramedics in Western Australia compared conventional paramedics with special operations paramedics (D. Chapman et al. 2007). Some information comparing their physical profiles is shown in Table 32.7.

Compute the missing standard errors.
Consider comparing the mean grip strength for the two groups of paramedics. The standard error for the difference between the means is 3.3044.
1. Carefully write down the hypotheses.
2. Compute the $t$-score for testing if a difference exists between the two types of paramedics.
3. Approximate the $P$-value using the 68--95--99.7 rule.
4. Discuss the conditions required for statistical validity in this context.
5. Make a conclusion.
Consider comparing the mean number of push-ups completed in one minute. The standard error for the difference between the means is 4.0689.
1. Carefully write down the hypotheses.
2. Compute the $t$-score for testing if a difference exists between the two types of paramedics.
3. Approximate the $P$-value using the 68--95--99.7 rule.
4. Discuss the conditions required for statistical validity in this context.
5. Make a conclusion.

TABLE 32.7: The physical profile of conventional ($n=18$) and special operation ($n = 11$) paramedics in Western Australia
	Conventional	Special Operations
Grip strength (in kg)
Mean	51	56
Std deviation	8	9
Std error
Push-ups (per minutes)
Mean	36	47
Std deviation	10	11
Std error

Exercise 32.4 Consider again the body temperature data from Sect. 29.1. The researchers also recorded the gender of the patients, as they also wanted to compare the mean internal body temperatures for males and females.

Use the jamovi output in Fig. 32.4 to perform this test and make a conclusion. Also comment on the practical significance of your results.

FIGURE 32.4: jamovi output for the body-temperature data

Exercise 32.5 (These data were also seen in Exercise 24.5.) A study (Woodward and Walker 1994) examined the sugar consumption in industrialised (mean: 41.8 kg/person/year) and non-industrialised (mean: 24.6 kg/person/year) countries. The jamovi output is shown in Fig. 32.5.

Write the hypotheses.
Write down and interpret the CI.
Write a conclusion for the hypothesis test.

FIGURE 32.5: jamovi output for the sugar-consumption data

Exercise 32.6 (These data were also seen in Exercise 24.6.) In an attempt to reduce vehicle speeds on freeway exit ramps, a Chinese study tried using additional signage (Ma et al. 2019). At one site studied (Ningxuan Freeway), speeds were recorded at various points on the freeway exit for vehicles before the extra signage was added, and then for different vehicles after the extra signage was added.

From this data, the deceleration of each vehicle was determined (data with Exercise 24.6) as the vehicle left the 120 km/h speed zone and approached the 80 km/hr speed zone. Use the data, and the summary in Table 32.8, to test the RQ:

At this freeway exit, is the mean vehicle deceleration the same before extra signage is added and after extra signage is added?

TABLE 32.8: The signage data summary (in m/s-squared)
	Mean	Std deviation	Sample size	Std error
Before	0.0745	0.0494	0.00802	38
After	0.0765	0.0521	0.00814	41
Change	-0.0020		0.00181

Exercise 32.7 (This study was seen in Exercise 24.7.) A study (Wojcik et al. 1999) compared the lean-forward angle in younger and older women. An elaborate set-up was constructed to measure this lean-forward angle, using harnesses. Consider this RQ:

Among healthy women, is the mean lean-forward angle greater for younger women compared to older women?

Use the software output (Fig. 32.6) to answer these questions:

What is the parameter? Carefully describe what it means.
Is the test one- or two-tailed?
Write the statistical hypothesis.
Use the jamovi output to conduct the hypothesis test.
Write a conclusion.

FIGURE 32.6: jamovi output for the face-plant data

31 Tests for the mean difference (paired data)

33 Tests for comparing odds

	Group A	Group B
Population means:	\(\mu_A\)	\(\mu_B\)
Sample means:	\(\bar{x}_A\)	\(\bar{x}_B\)
Standard deviations:	\(s_A\)	\(s_B\)
Standard errors:	\(\displaystyle\text{s.e.}(\bar{x}_A) = \frac{s_A}{\sqrt{n_A}}\)	\(\displaystyle\text{s.e.}(\bar{x}_B) = \frac{s_B}{\sqrt{n_B}}\)
Sample sizes:	\(n_A\)	\(n_B\)

Scientific Research and Methodology: An introduction to quantitative research and statistics

32 Tests for means of two independent groups

32.1 Introduction: reaction times

32.2 Hypotheses and notation

32.3 Describing the sampling distribution

32.4 Computing the test statistic

32.5 Determining \(P\)-values

32.6 Writing conclusions

32.7 Statistical validity conditions

32.8 Example: speed signage

32.9 Example: health promotion services

32.10 Summary

32.11 Quick review questions

32.12 Exercises