39 Regression

So far, you have learnt about the research process, including analysing data using confidence intervals and hypothesis tests. In the last chapter, you were introduced to the correlation coefficient for quantifying the strength and direction of a linear relationship between two quantitative variables.

In this chapter, you will learn to:

• produce and interpret linear regression equations.
• conduct hypothesis tests for the slope in a regression line.
• produce confidence intervals for the slope in a regression line.

39.1 Introduction

Chapters 17 and 38 introduced correlation, which measures the strength of the linear relationship between two quantitative variables \(x\) (an explanatory variable) and \(y\) (a response variable). We now study regression, which describes what the linear relationship is between \(x\) and \(y\). The relationship is described mathematically, using an equation, which allows us to:

1. Predict values of \(y\) from given values of \(x\) (Sect. 39.4); and
2. Understand the relationship between \(x\) and \(y\) (Sect. 39.5).

An example of a linear regression equation, describing the relationship between the observed values of an explanatory variable \(x\) and the observed values of a response variable \(y\), is
\[ \hat{y} = -4 + (2\times x), \qquad\text{usually written}\qquad \hat{y} = -4 + 2x. \] The notation \(\hat{y}\) refers to the mean predicted value of the response variable \(y\), given a value of \(x\).

\(y\) refers to the values of the response variable observed from individuals. \(\hat{y}\) refers to mean predicted (not observed) values of \(y\) for some given value of \(x\).

\(\hat{y}\) is pronounced as 'why-hat'; the 'caret' above the \(y\) is called a 'hat', and is used to denote a predicted value (of \(y\)).

More generally, the equation of a straight line is
\[ \hat{y} = b_0 + (b_1 \times x), \qquad\text{usually written}\qquad \hat{y} = b_0 + b_1 x \] where \(b_0\) and \(b_1\) are (unknown) numbers estimated from sample data. Notice that \(b_1\) is the number multiplied by \(x\). In the example above, \(b_0 = -4\) and \(b_1 = 2\).

Example 39.1 (Regression equations) A report on the growth of Australian children (Pfizer Australia 2008) found an approximate linear relationship between the age of girls in years \(x\) and their height in cm \(y\). For girls between four and seven years of age, the regression equation was approximately
\[ \hat{y} = 73 + 7x. \] In this equation, \(b_0 = 73\) and \(b_1 = 7\).

Consider the regression equation \(\hat{y} = -0.0047x + 2.1\). What are the values of \(b_0\) and \(b_1\)? (Be careful!)

\(b_0 = 2.1\) and \(b_1 = -0.0047\). Remember: \(b_1\) is the number multiplied by the \(x\), and that \(b_0\) is the number by itself.

Some software and some journal papers write regression equations with the value of \(b_0\) first, and some with the value of \(b_1\) first. The order is not important.

39.2 Linear equations: review

To introduce, or revise, the idea of a linear equation, consider the (artificial) data in Fig. 39.1, with an explanatory variable \(x\) and a response variable \(y\). In the graph, I have drawn a sensible line on the graph that seems to capture the relationship between \(x\) and \(y\) (your line may look a little different).

To describe the line, a regression equation is used. In the regression equation \(\hat{y} = b_0 + b_1 x\), the numbers \(b_0\) and \(b_1\) are called regression coefficients, where

• \(b_0\) is called the intercept or the \(y\)-intercept. Its value corresponds to the predicted mean value of \(y\) when \(x = 0\).
• \(b_1\) is the slope. It measures how much the value of \(y\) changes, on average, when the value of \(x\) increases by 1.

We will use software to find the values of \(b_0\) and \(b_1\) (the formulas are tedious to use). However, a rough approximation of the value of \(b_0\) and \(b_1\) can be obtained by first drawing a sensible-looking straight line through the scatterplot of the data (Fig. 39.1).

A rough approximation of the value of the intercept \(b_0\) is made by noting what value of \(y\) the line predicts when \(x = 0\). When \(x = 0\), the regression line predicts the value of \(y\) to be about \(2\), so the value of \(b_0\) is approximately \(2\).

A rough approximation of the slope \(b_1\) is found by computing
\[ \text{approximate slope} = \frac{\text{Change in $y$}}{\text{Corresponding increase in $x$}} = \frac{\text{rise}}{\text{run}}. \] This approximation of the slope is the change in the value of \(y\) (the 'rise') divided by the corresponding increase in the value of \(x\) (the 'run').

FIGURE 39.1: An example scatterplot

To approximate the slope, the 'rise over run' idea is used; the animation below may help explain. When the value of \(x\) increases from \(1\) to \(5\) (an increase of \(5 - 1 = 4\)), the corresponding value of \(y\) changes from \(5\) to \(17\) (a change of \(17 - 5 = 12\)). So:
\[ \frac{\text{rise}}{\text{run}} = \frac{17 - 5}{5 - 1} = \frac{12}{4} = 3. \] The value of \(b_1\) is about \(3\). The regression line is approximately \(\hat{y} = 2 + (3\times x)\), usually written
\[ \hat{y} = 2 + 3x. \]

The regression equation has \(\hat{y}\) (not \(y\)) on the left-hand side. That is, the equation predicts the mean values of \(y\), which may not be equal to the observed values (which are denoted by \(y\)).

A 'good' regression equation would produce predicted values \(\hat{y}\) close to the observed values \(y\); that is, the lines passes close to each point on the scatterplot.

The intercept \(b_0\) has the same measurement units as the response variable. The measurement unit for the slope \(b_1\) is the 'measurement units of the response variable', per 'measurement units of the explanatory variable'.

Example 39.2 (Measurement units of regression parameters) In Example 39.1, the regression line for the relationship between the age of Australian girls in years \(x\) and their height in cm \(y\) was \(\hat{y} = 73 + 7x\) (for girls between four and seven years).

In this equation, the intercept is \(73\) cm and the slope is \(73\) cm/year.

Example 39.3 (Finding a rough approximation of the regression equation) For the red-deer data used in the previous chapters, a rough estimate of the regression line can be drawn on a scatterplot to estimate \(b_0\) and \(b_1\) (Fig. 39.2). The estimate of \(b_0\) is roughly \(4.4\). The estimate of \(b_1\), using the 'rise-over-run' idea, is \(-2/10 = -0.2\). This is a negative rise, since the value of \(y\) is smaller as the value of \(x\) increases. (The relationship is negative, so the slope is negative.) The rough guess of the regression line is
\[ \hat{y} = 4.4 - 0.2x, \] where \(x\) is the age (in years), and \(y\) is molar weight (in g). The rough guess of the intercept \(b_0\) is \(4.4\) g, while the rough guess of the slope \(b_1\) is \(-0.2\) g/year.

FIGURE 39.2: Obtaining rough guesses for the regression equation for the red-deer data. Left: Approximating \(b_0\). Right: approximating \(b_1\) using rise-over-run

You may like the following interactive activity, which explores slopes and intercepts.

Example 39.4 (Estimating regression parameters) The relationship (P. K. Dunn and Smyth 2018) between the number of cyclones \(y\) in the Australian region each year from 1969 to 2005, and a climatological index called the Ocean Niño Index (ONI, \(x\)), is shown in Fig. 39.3.

When the value of \(x\) is zero, the predicted value of \(y\) is about \(12\); \(b_0\) is about \(12\). (You may get something slightly different.) Notice that the intercept is the predicted value of \(y\) when \(x = 0\), which is not at the left of the graph.

To approximate the value of \(b_1\), use the 'rise over run' idea. When \(x\) is about \(-2\), the predicted value of \(y\) is about \(17\); when \(x\) is about \(2\), the predicted value of \(y\) is about \(8\). The value of \(x\) increases by \(2 - (-2) = 4\), while the value of \(y\) changes by \(8 - 17 = -9\) (a decrease of about \(9\)). Hence, \(b_1\) is approximately \(-9/4 = -2.25\). (You may get something slightly different.) The relationship has a negative direction, so the slope must be negative. The regression line is approximately \(\hat{y} = 12 - 2.25x\).

FIGURE 39.3: The number of cyclones in the Australian region each year from 1969 to 2005, and the ONI for October, November, December

The above method gives a crude approximation to the values of the intercept \(b_0\) and the slope \(b_1\). In practice, many reasonable lines could be drawn through a scatterplot of data. However, one of those lines is the 'best' line in some sense¹⁵. Software calculates this 'line of best fit' for us.

39.3 Fitting regression equations using software

For the red-deer data again (Fig. 17.2), part of the relevant jamovi output is shown in Fig. 39.4. From the output, the sample slope is \(b_1 = -0.181\), and the sample \(y\)-intercept is \(b_0 = 4.398\): the values of \(b_0\) and \(b_1\) are in the column labelled Estimate in jamovi. These are the values of the two regression coefficients; so
\[ \hat{y} = 4.398 + (-0.181\times x), \qquad\text{usually written as}\qquad \hat{y} = 4.398 - 0.181 x. \] These are close to the values obtained using the rough method in Sect. 39.2.

The sign of the slope \(b_1\) and the sign of correlation coefficient \(r\) are always the same. For example, if the slope is negative, the correlation coefficient will also be negative.

FIGURE 39.4: Output for the red-deer data

Example 39.5 (Regression coefficients) The regression equation for the cyclone data (Fig. 39.3) is found from the jamovi output (Fig. 39.5):
\[ \hat{y} = 12.14 - 2.23x, \] where \(x\) is the ONI (averaged over October, November, December) and \(y\) is the number of cyclones. These values are close the approximations made in Example 39.4 (\(12\) and \(-2.25\) respectively).

FIGURE 39.5: Output for the cyclone data

You may like the following interactive activity, which explores regression equations.

39.4 Using regression equation for making predictions

The regression equation for the red-deer data, estimated from one of the many possible samples, is \(\hat{y} = 4.398 - 0.181 x\), and can be used to make predictions of the mean value of \(y\) for a given value of \(x\). For example, the equation can be used to predict the average molar weight for deer \(10\) years old. Since \(x\) represents the age, use \(x = 10\) in the regression equation:
\[\begin{align*} \hat{y} &= 4.398 - (0.181\times 10)\\ &= 4.398 - 1.81 = 2.588. \end{align*}\] Male red deer aged \(10\) years are predicted to have a mean molar weight of \(2.588\) g. Some individual deer aged \(10\) will have molars weighing more than this, and some weighing less than this; the model predicts that the mean molar weight for male red deer aged \(10\) will be about \(2.588\) g.

(Note: The value of \(\hat{y}\) is computed using the estimates \(b_0\) and \(b_1\), which are in turn computed from sample data. Hence, the value of \(\hat{y}\) is also depends on which one of the countless possible samples is used, so also has a standard error.)

What is the predicted mean molar weight for for male red deer \(12\) years of age?

Prediction: \(4.398 - (0.181 \times 12) = 2.226\), or about \(2.23\) g.

Suppose we were interested in male red deer \(20\) years of age; the mean predicted weight of the molars is \(4.398 - (0.181 \times 20) = 0.778\), or about \(0.78\) g. This prediction may be useful prediction... but it may be rubbish.

In the data, the oldest deer is aged \(14.4\) years, so the regression line may not even apply for deer aged over \(14.4\) years of age. For example, the relationship may be non-linear after \(14\) years of age, or red deer may not even live to \(20\) years of age. The prediction may be sensible... but it may not be either. We don't know whether the prediction is sensible or not, because we have no data for deer aged over \(14.4\) years to inform us. Making predictions outside the range of the available data is called extrapolation, and extrapolation beyond the data may lead to nonsense predictions.

Definition 39.1 (Extrapolation) Extrapolation refers to making predictions outside the range of the available data. Extrapolation beyond the data may lead to nonsense predictions.

39.5 Using regression equations for understanding relationships

The regression equation can be used to understand the relationship between the two variables. Consider again the red-deer regression equation:
\[\begin{equation} \hat{y} = 4.398 - 0.181 x. \tag{39.1} \end{equation}\] What does this equation reveal about the relationship between \(x\) and \(y\)?

39.5.1 The meaning of \(b_0\)

\(b_0\) is the predicted value of \(y\) when \(x = 0\). Using \(x = 0\) in Eq. (39.1) predicts a mean molar weight of
\[ \hat{y} = 4.398 - (0.181\times 0) = 4.398 \] for deer zero years of age (i.e., newborn male red deer). This prediction is the predicted mean molar weight; some individual deer will have molars weights greater than this, and some less than this. But in any case, this predicted mean may be nonsense: it is extrapolating beyond the data (the youngest deer in the sample is aged \(4.4\) years). Indeed, young deer are born with molars that are replaced as they grow.

The value of the intercept \(b_0\) is sometimes (but not always) meaningless. The value of the slope \(b_1\) is usually of greater interest, as it explains the relationship between the two variables.

39.5.2 The meaning of \(b_1\)

\(b_1\) tells us how much the value of \(y\) changes (on average) when the value of \(x\) increases by one. For the red-deer data, \(b_1\) tells us how much the molar weight changes (on average) when age increases by one year. Specifically, each extra year older is associated with an average change of \(-0.181\) g in molar weight; that is, a decrease in molar weight by a mean of \(0.181\) g. The molars of some individual deer will lose more weight than this in some years, and some will lose less... the value is a mean weight loss per year.

To demonstrate: When \(x = 10\), \(y\) is predicted to be \(\hat{y}= 2.588\) g. For deer one year older than this (i.e. \(x = 11\)) we predict \(y\) to increase by \(b_1 = -0.181\) g (or, equivalently, decrease by \(0.181\) g). That is, we would predict \(\hat{y}= 2.588 - 0.181 = 2.407\) g. This is the same prediction made by using \(x = 11\) in Eq. (39.1).

If the value of \(b_1\) is positive, then the predicted values of \(y\) increase as the values of \(x\) increase. If the value of \(b_1\) is negative, then the predicted values of \(y\) decrease as the values of \(x\) increase.

This interpretation of \(b_1\) explains the relationship: each extra year of age reduces the weight of the molars by \(0.181\) g, on average, in male red deer. Recall that the units of measurement of the slope here are 'grams per year'.

If the value of the slope is zero, there is no linear relationship between \(x\) and \(y\). In this case, the correlation coefficient is also zero.

39.6 Confidence intervals for the regression parameters

39.6.1 Introduction

The regression line is computed from the observed sample data. A regression equation exists in the population that is estimated using the sample information. In the population, the intercept is denoted by \(\beta_0\) and the slope by \(\beta_1\). These population values are unknown, and are estimated by the statistics \(b_0\) and \(b_1\) respectively.

The symbol \(\beta\) is the Greek letter 'beta', pronounced 'beater' (as in 'egg beater'). So \(\beta_0\) is usually pronounced as 'beater-zero', and \(\beta_1\) as 'beater-one'.

Every sample is likely to produce a slightly different value for both \(b_0\) and \(b_1\) (sampling variation), so both \(b_0\) and \(b_1\) have a standard error. The formulas for computing the values of \(b_0\) and \(b_1\) (and their standard errors) are intimidating, so we will allow software to do the calculations.

The sample estimates can be used to ask questions about the unknown population regression coefficients. As usual, the sample estimates can vary across all possible samples (and so have a sampling distribution). Hence, computing confidence intervals for \(\beta_0\) and \(\beta_1\) makes sense.

Usually the slope of greater interest than the intercept, because the slope explains the relationship between the two variables (Sect. 39.5), so forming confidence intervals for \(\beta_1\) is more common than for \(\beta_0\), though the same ideas apply.

Using the output (Fig. 39.4) for the red-deer data, can you determine the approximate \(95\)% CI for \(\beta_1\)?

39.6.2 Describing the sampling distribution

The observed value of the slope will vary across all possible samples; sampling variation is present. However, under certain conditions (Sect. 39.8), the sampling distribution can be described using a normal distribution.

Definition 39.2 (Sampling distribution of a sample slope) The sampling distribution of a sample regression slope is (when certain conditions are met; Sect. 39.8) described by

• an approximate normal distribution,
• with a mean of \(\beta_1\), and
• a standard deviation, called the standard error of the slope and denoted \(\text{s.e.}(b_1)\).

(A formula exists for finding \(\text{s.e.}(b_1)\), but it is tedious to use and we will not give it.)

This sampling distribution describes all possible values of the sample slope from all possible samples, through sampling variation. For the red-deer data then, the values of the sample slope across all possible samples is described (Fig. 39.6) as:

• an approximate normal distribution,
• with a sampling mean whose values is \(\beta_1\), and
• with a standard deviation, called the standard error of the slope \(\text{s.e.}(b_1) = 0.029\) (from software; Fig. 39.4).

FIGURE 39.6: The distribution of sample slopes for the red-deer data, around the true slope \(\beta_1\)

As seen earlier, most CIs have the form
\[ \text{statistic} \pm ( \text{multiplier} \times \text{standard error}), \] where the multiplier is \(2\) for an approximate \(95\)% CI (from the \(68\)--\(95\)--\(99.7\) rule). Using the standard error reported by software, an approximate \(95\)% CI is \(-0.181 \pm (2\times 0.029)\), or \(-0.181 \pm 0.058\), or from \(-0.239\) to \(-0.123\) g/year.

Software can be used to produce exact CIs too (Fig. 39.7). The approximate and exact \(95\)% CIs are very similar. We could write:

For each increase of one year in the age of male red deer, the mean molar weight increases by \(-0.181\) grams per year (approx. \(95\)% CI: \(-0.239\) to \(-0.123\)).

Alternatively (and equivalently):

For each increase of one year in the age of male red deer, the mean molar weight decreases by \(0.181\) grams per year (approx. \(95\)% CI: \(0.239\) to \(0.123\)).

FIGURE 39.7: Output for the red-deer data, including the CIs for the regression parameters

Example 39.6 (Cyclones) Using the jamovi output (Fig. 39.5) for the cyclones data, we can see that the approximate \(95\)% CI for the regression slope \(\beta_1\) is \[ -2.23 \pm (2 \times 0.404),\text{\qquad {or}\qquad} -2.23 \pm 0.808. \] The interval is from \(-3.04\) to \(-1.42\).

39.7 Hypothesis testing for the regression parameters

39.7.1 Statistical hypotheses

Since the regression line is computed from one of countless possible samples, any relationship between \(x\) and \(y\) observed in the sample may be due to sampling variation. Possibly, no relationship exists in the population; the only reason why a relation appears in the sample is due to sampling variation. That is, a hypothesis test can be conducted.

The null hypothesis is the usual 'no relationship' hypothesis. In this context, 'no relationship' means that the slope is zero (Sect. 39.5.2), so the null hypotheses (about the population) is \(H_0\): \(\beta_1 = 0\). Observe what happens if the slope is zero. Since \(beta_1\) is the change in \(y\) (on average) when \(x\) increase by one, \(\beta_1 = 0\) means that the predicted mean value of \(y\) changes by zero if the value of \(x\) changes by one. In other words, if the value of \(x\) changes, the predicted mean value of \(y\) doesn't change. This is equivalent to saying that there is no relationship between the variables. (We would also find \(r = 0\).)

As part of the decision-making process, the null hypothesis is initially assumed to be true. For the red-deer data, the RQ posed in Sect. 38.2 is

In male red deer, does the mean molar weight decrease as deer age?

This RQ implies these hypotheses:
\[ \text{$H_0$: } \beta_1 = 0;\quad\text{and}\quad\text{$H_1$: } \beta_1 < 0. \] The parameter is \(\beta_1\), the population slope for the regression equation predicting molar weight from age. The alternative hypothesis is one-tailed, based on the RQ (see Sect. 38.2).

39.7.2 Describing the sampling distribution

Assuming the null hypothesis is true (that \(\beta_1 = 0\)), the possible values of the sample slope \(b_1\) can be described, due to sampling variation (based on Def. 39.2 by setting \(\beta_1 = 0\)). The variation in the sample slope across all possible samples when \(\beta_1 = 0\) is described (Fig. 39.8) using:

• an approximate normal distribution,
• with a sampling mean whose value is \(\beta_1 = 0\) (from \(H_0\)), and
• a standard deviation, called the standard error of the slope and denoted \(\text{s.e.}(b_1)\), with a value of \(0.0289\) (from software; Fig. 39.7).

FIGURE 39.8: The distribution of sample slopes for the red-deer data, if the population slope is \(\beta_1 = 0\)

39.7.3 Computing the test statistic

The observed sample slope is \(b_1 = -0.181\). Locating this on Fig. 39.8 (way to the left) shows that it is very unlikely that one of the many possible samples would produce such a slope, just through sampling variation, if the population slope really was \(\beta_1 = 0\). The test statistic is found using the usual approach when the sampling distribution has an approximate normal distribution:
\[\begin{align*} t &= \frac{\text{observed value} - \text{mean of the statistic's distribution}}{\text{standard dev. of the statistics distribution}}\\ &= \frac{ b_1 - \beta_1}{\text{s.e.}(b_1)} = \frac{-0.181 - 0}{0.0289} = -6.27, \end{align*}\] where the values of \(b_1\) and \(\text{s.e.}(b_1)\) are taken from the software output (Fig. 39.7). This \(t\)-score is the same value reported by the software.

39.7.4 Determining the \(P\)-value

To determine if the statistic is consistent with the null hypothesis, the \(P\)-value can be approximated using the \(68\)--\(95\)--\(99.7\) rule, approximated using tables, or taken from software output (Fig. 39.7). The \(P\)-value will be very small; software shows the one-tailed \(P\)-value is \(P < 0.0005\).

39.7.5 Writing conclusions

We write:

The sample presents very strong evidence (\(t = -6.27\); one-tailed \(P < 0.0005\)) that, in the population, the mean weight of molars decreases as male red deer get older (slope: \(-0.181\); \(95\)% CI from \(-0.239\) to \(-0.124\)).

Notice the three features of writing conclusions again: An answer to the RQ; evidence to support the conclusion (\(t = -6.27\); one-tailed \(P < 0.0005\)); and some sample summary information (slope: \(-0.181\); \(95\)% CI from \(-0.239\) to \(-0.124\)).

Example 39.7 (Emergency department patients) For the cyclone data, the relationship between the number of cyclones and the ONI is being examined. The RQ is

In the Australian region, is there a relationship between ONI and the number of cyclones?

This RQ implies these hypotheses:
\[ \text{$H_0$: } \beta_1 = 0;\quad\text{and}\quad\text{$H_1$: } \beta_1 \ne 0. \] From the jamovi output (Fig. 39.5), \(t = -5.52\) and the \(P\)-value is small as expected: \(P < 0.001\).

We write:

The sample presents very strong evidence (\(t = -5.52\); two-tailed \(P < 0.001\)) that, in the population, the mean number of cyclones is related to the ONI (slope: \(-2.23\); \(95\)% CI from \(-3.04\) to \(-1.42\)).

39.8 Statistical validity conditions

The results for the CI and the hypothesis test hold when certain conditions are met. The conditions for which the test is statistically valid are the same as for correlation (Sect. 38.3):

1. The relationship is approximately linear.
2. The variation in the response variable is approximately constant for all values of the explanatory variable.
3. The sample size is at least \(25\).

The sample size of \(25\) is a rough figure; some books give other values.

Example 39.8 (Statistical validity) For the red-deer data, the relationship is approximately linear, and the variation in molar weight appears to be somewhat constant for various ages (Fig. 17.2), so regression is appropriate. The sample size is \(n = 78\). The hypothesis test is statistically valid.

Example 39.9 (Cyclones) The scatterplot for the cyclones data (Fig. 39.3) shows the relationship is approximately linear, that the variation in the number of cyclones seems reasonably constant for different values of the ONI, and the sample size is larger than \(25\) (\(n = 37\)). The CI (Example 39.6) and the test (Example 39.7) is statistically valid.

39.9 Example: removal efficiency

(This study was seen in Sect. 38.4.) In wastewater treatment facilities, air from biofiltration is passed through a membrane and dissolved in water, and is transformed into harmless by-products. The removal efficiency \(y\) (in percent) may depend on the inlet temperature (in \(^\circ\)C; \(x\)). A RQ is

In treating biofiltration wastewater, is removal efficiency related to the inlet temperature?

The scatterplot of the data (Fig. 38.3) shows the relationship is approximately linear. From the software output (Fig. 39.10), \(b_0 = 97.5\) and \(b_1 = 0.0757\), so the regression equation is
\[ \hat{y} = 97.5 + 0.0757x \] for \(x\) and \(y\) defined above. The slope quantifies the relationship, so we can test
\[ \text{$H_0$: } \beta_1 = 0 \qquad\text{and}\qquad \text{$H_1$: } \beta_1 \ne 0. \] From the output, \(t = 10.7\) which is huge, and the \(P\)-value is small as expected: \(P < 0.001\). The output does not include the CI, but since \(\text{s.e.}(b_1) = 0.00705\), the approximate \(95\)% CI is
\[ 0.0757 \pm (2 \times 0.00705), \quad\text{ or }\quad 0.0757 \pm 0.0141. \] We can write:

Very strong evidence exists (\(t = 10.7\); \(P < 0.001\)) that inlet temperature is related to removal efficiency (slope: \(0.0757\); approximate \(95\)% CI: \(0.0616\) to \(0.0898\)).

FIGURE 39.9: The relationship between removal efficiency and inlet temperature

FIGURE 39.10: Regression output for the removal-efficiency data

39.10 Chapter summary

In this chapter, we have learnt about regression, which mathematically describes the relationship between two quantitative variables. The response variable is denoted by \(y\), and the explanatory variable by \(x\). The linear relationship between them (the regression equation), in the sample, is
\[ \hat{y} = b_0 + b_1 x, \] where \(b_0\) is a number (the intercept), \(b_1\) is a number (the slope), and the 'hat' above the \(y\) indicates that the equation gives an predicted mean value of \(y\) for the given \(x\) value.

The intercept is the predicted mean value of \(y\) when the value of \(x\) is zero. The slope is how much the predicted mean value of \(y\) changes, on average, when the value of \(x\) increases by \(1\).

The regression equation can be used to make predictions or to understand the relationship between the two variables. Predictions made with values of \(x\) outside the values of \(x\) used to create the regression equation (called extrapolation) may not be reliable.

In the population, the regression equation is
\[ \hat{y} = \beta_0 + \beta_1 x. \] To test a hypothesis about a population slope \(\beta_1\), based on the value of the sample slope \(b_1\), assume the value of \(\beta_1\) in the null hypothesis (usually zero) to be true. Then, the sample slope varies from sample to sample and, under certain statistical validity conditions, varies with an approximate normal distribution centred around the hypothesised value of \(\beta_1\), with a standard deviation of \(\text{s.e.}(b_1)\). This distribution describes what values of the sample slope could be expected in the sample if the value of \(\beta_1\) in the null hypothesis was true.

The test statistic is
\[ t = \frac{ b_1 - \beta_1}{\text{s.e.}(b_1)}, \] where \(\beta_1\) is the hypothesised value given in the null hypothesis (usually zero). The \(t\)-value is like a \(z\)-score, and so an approximate \(P\)-value can be estimated using the \(68\)--\(95\)--\(99.7\) rule.

The following short video may help explain some of these concepts:

39.11 Quick review questions

A study of athletes (Telford and Cunningham 1991) examined the relationship between the height and weight of \(n = 37\) rowers at the Australian Institute of Sport (AIS), as shown in Fig. 39.11.

FIGURE 39.11: Scatterplot of Weight against Height rowers at the AIS

1. What is the \(y\)-variable? Weight Height
2. Using the 'rise over run' idea, what is the approximate value of the slope? -140 1 -1 50
3. The regression equation is \(\hat{y} = -138 + 1.2 x\). What does \(x\) represent?
   The weight of the rowersThe height of the rowers
4. What does the 'hat' above the \(y\) mean?
   That the weights are not measured accuratelyThat the weights are population valuesThat the regression model gives poor estimatesThat the regression model gives good estimatesThat the regression model gives an estimate of the weight
5. To two decimal places, what weight would be predicted for a rower who is \(180\) cm tall?
6. The standard error of the slope is \(0.112\). What is the value of the test statistic (to one decimal place) to test if the population slope is zero?
7. True or false? The \(P\)-value for this test will be very small. TRUE FALSE
8. True or false? The units of the slope are kg/cm. TRUE FALSE
9. True or false? Making a prediction for the weight of a rower weighing \(220\) kg would be an example of extrapolation. TRUE FALSE

39.12 Exercises

Answers to odd-numbered exercises are available in App. E.

Exercise 39.1 [Dataset: Throttle] A study (Amin and Mahmood-ul-Hasan 2019) of gas engines measured the throttle angle (\(x\)) and the manifold air pressure (\(y\)) as a fraction of the maximum value.

1. The value of \(r\) is given in the paper as \(0.972986604\). Comment on this value, and what it means.
2. Comment on the use of a regression model, based on the scatterplot (Fig. 39.12).
3. The authors fitted the following regression model: \(y = 0.009 + 0.458x\). Identify errors that the researchers have made when giving this regression equation (there are many).
4. Critique the researchers' approach.

FIGURE 39.12: Manifold air pressure plotted against throttle angle for an internal-combustion gas engine

Exercise 39.2 [Dataset: OSA] In a study of obstructive sleep apnoea (OSA) in adults with Down Syndrome (de Carvalho et al. 2020), \(n = 60\) adults underwent a sleep study and had various information recorded. The response variable was OSA severity: the average number of episodes of sleep disruption (according to specific criteria) per hour of sleep (called the Respiratory Event Index, REI). One research question is

Among Down Syndrome adults, is there a relationship between the REI and neck size?

Here, \(x\) is the neck size (in cm), and \(y\) is the REI value. The data are shown in Fig. 39.14.

1. Using the jamovi output (Fig. 39.13, right panel), determine the value of the correlation coefficient.
2. Compute the value of \(R^2\), and interpret what this means.
3. Using the jamovi, write down the values of the intercept and the slope, and hence the regression equation.
4. Interpret what the slope in the regression equation means.
5. Find a CI for the slope.
6. Perform a hypothesis to test if a relationship exists between the variables.
7. Are the test and CI statistically valid?

FIGURE 39.13: Left: Scatterplot of the neck circumference vs REI for Down Syndrome adults. Right: jamovi output.

FIGURE 39.14: The obstructive sleep apnoea dataset

Exercise 39.3 [Dataset: EDpatients] A study examined the relationship between the number of emergency department (ED) patients and the number of days following the distribution of monthly welfare monies (Brunette, Kominsky, and Ruiz 1991) from 1986 to 1988 in Minneapolis, USA. The data (extracted from Fig. 2 of Brunette, Kominsky, and Ruiz (1991)) are displayed in Fig. 39.15 (left panel), and the jamovi output in Fig. 39.15 (right panel)).

1. Write down the estimated regression equation.
2. Interpret the slope in the regression equation.
3. Find a \(95\)% CI for the slope.
4. Conduct a hypothesis test for the slope, and explain what the results mean.

FIGURE 39.15: The number of emergency department patients, and the number of days since distribution of welfare. Left: scatterplot. Right: jamovi output.

Exercise 39.4 [Dataset: Punting] (These data were also seen in Exercise 38.7.) A study (Myers (1990), p. 75) of American footballers measured the right-leg strengths \(x\) of \(13\) players (using a weight lifting test), and the distance \(y\) they punt a football (with their right leg).

1. Use the plot (Fig. 39.16, left panel) to approximate of the values of the intercept and slope.
2. Using the jamovi output (Fig. 39.16, right panel), write down the value of the slope (\(b_1\)) and \(y\)-intercept (\(b_0\)).
3. Hence write down the regression equation.
4. Interpret the slope (\(b_1\)).
5. Write the hypotheses for testing for a relationship in the population
6. Write down the \(t\)-score and \(P\)-value from the output.
7. Determine an approximate \(95\)% CI for the population slope \(\beta_1\).
8. Write a conclusion.

FIGURE 39.16: Punting distance and right leg strength

Exercise 39.5 [Dataset: Gorillas] (These data were also seen in Exercise 17.8.) A study (Wright et al. 2021) examined \(25\) gorillas and recorded information about their chest beating and their size (measured by the breadth of the gorillas' backs). The relationship is shown in a scatterplot in Fig. 17.12 (left panel).

Use the output (Fig. 39.17) to find the regression line, and perform a hypothesis tests for the slope. Write a conclusion.

FIGURE 39.17: jamovi regression output for the gorilla data

Exercise 39.6 [Dataset: Bitumen] (These data were also seen in Exercise 38.6.) A study of hot mix asphalt (Panda, Das, and Sahoo 2018) created \(n = 42\) samples of asphalt and measured the volume of air voids and the bitumen content by weight (Fig. 39.18). Use the software output (Fig. 39.19) to answer these questions.

1. Write down the regression equation.
2. Interpret what the regression equation means.
3. Perform a test to determine is there is a relationship between the variables.
4. Predict the mean percentage of air voids by volume when the percentage bitumen is \(5.0\)%. Do you expect this to be a good prediction? Why or why not?
5. Predict the mean percentage of air voids by volume when the percentage bitumen is \(6.0\)%. Do you expect this to be a good prediction? Why or why not?

FIGURE 39.18: Air voids in bitumen samples

FIGURE 39.19: jamovi regression output for the bitumen data

Exercise 39.7 [Dataset: Possums] (These data were also seen in Exercise 38.9.) A study of Leadbeater's possums in the Victorian Central Highlands (J. L. Williams et al. 2022) recorded, among other information, the body weight of the possums and their location, including the elevation of the location. A scatterplot of the data is shown in Fig. 38.9. The jamovi output for fitting the regression line is shown in Fig. 39.20.

1. Write down the regression equation.
2. Interpret what the regression equation means.
3. Perform a test to determine is there is a relationship between the possum weight and the elevation.
4. Interpret the meaning of the slope.
5. Predict the mean weight of male possums at an elevation of \(1000\) m. Do you expect this to be a good prediction? Why or why not?
6. Predict the mean weight of male possums at an elevation of \(200\) m. Do you expect this to be a good prediction? Why or why not?

FIGURE 39.20: The relationship between weight of possums and the elevation of their location

Exercise 39.8 [Dataset: Soils] (These data were also seen in Exercise 38.10.) The California Bearing Ratio (CBR) value is used to describe soil-sub grade for flexible pavements (such as in the design of air field runways). One study (Talukdar 2014) examined the relationship between CBR and other properties of soil, including the plasticity index (PI, a measure of the plasticity of the soil). The scatterplot from \(16\) different soil samples from Assam, India, is shown in Fig. 38.10.

Estimate the regression equation, using the 'rise-over-run' idea.

Exercise 39.9 A study of \(n = 31\) people on the use of sunscreen (Heerfordt et al. 2018) explored the relationship between the time (in minutes) spent on sunscreen application \(x\) and the amount (in grams) of sunscreen applied (\(y\)). The fitted regression equation was \(\hat{y} = 0.27 + 2.21x\).

1. Interpret the meaning of \(b_0\) and \(b_1\). Do they seems sensible?
2. According to the article, a hypothesis for testing \(\beta_0\) produced a \(P\)-value much larger than \(0.05\). What does this mean?
3. If someone spent \(84\) mins applying sunscreen, how much sunscreen would you predict that they used?
4. The article reports that \(R^2 = 0.64\). Interpret this value.
5. What is the value of the correlation coefficient?

Exercise 39.10 One study (Bhargava et al. 1985) stated (p. 1617):

In developing countries [...] logistic problems prevent the weighing of every newborn child. A study was performed to see whether other simpler measurements could be substituted for weight to identify neonates of low birth weight and those at risk.

The relationship between infant chest circumference (in cm) \(x\) and birth weight (in grams) \(y\) was given as:
\[ \hat{y} = -3440.2403 + 199.2987x. \] The correlation coefficient was \(r = 0.8696\) with \(P < 0.001\).

1. Critique the way in which the regression equation and correlation coefficient are reported.
2. Based on the regression equation only, could chest circumference be used as a useful predictor of birth weight? Explain.
3. Based on the correlation information only, could chest circumference be used as a useful predictor of birth weight? Explain.
4. Interpret the intercept and the slope of the regression equation.
5. What units of measurement are the intercept and slope measured in?
6. Predict the birth weight of an infant with a chest circumference of 30cm.

---

Peter K. Dunn, 2024: CC BY-NC-SA 4.0