36 Regression

So far, you have learnt to ask a RQ, identify different ways of obtaining data, design the study, collect the data describe the data, summarise data graphically and numerically, understand the tools of inference, to form confidence intervals, and to perform hypothesis tests.

In this chapter, you will learn about regression. You will learn to:

  • produce and interpret linear regression equations.
  • conduct hypothesis tests for the slope in a regression line.
  • produce confidence intervals for the slope in a regression line.

36.1 Introduction

In the last chapter, correlation was studied, which measures the strength of the linear relationship between two quantitative variables \(x\) and \(y\). We now study regression, which describes what the linear relationship is between \(x\) and \(y\).

The relationship is described using an equation, which allows us to:

  1. Predict values of \(y\) from given values of \(x\) (Sect. 36.4); and
  2. Understand the relationship between \(x\) and \(y\) (Sect. 36.5).

36.2 Linear equations: A review

An example of a regression equation is

\[ \hat{y} = -4 + 2x. \] Here, \(x\) refers to the explanatory variable, \(y\) refers to the observed response variable, and \(\hat{y}\) refers to the predicted values of the response variable.

In general, the equation of a straight line is written as

\[ \hat{y} = {b_0} + {b_1} x \] where \(b_0\) and \(b_1\) are just numbers. Again, \(\hat{y}\) refers to the predicted (not observed) values of \(y\).

\(\hat{y}\) is pronounced as 'why hat'; the 'caret' above the \(y\) is called a 'hat', and designates a predicted value (of \(y\)).

Example 36.1 (Regression equations) In the regression equation \(\hat{y} = 15 - 102x\), we have \(b_0=15\) and \(b_1 = -102\).

Consider the regression equation \(\hat{y} = -0.0047x + 2.1\).

What are the values of \(b_0\) and \(b_1\)? (Look carefully!)

(Answer is here536.)

The numbers \(b_0\) and \(b_1\) are called regression coefficients, where

  • \(b_0\) is a number called the intercept. It is the predicted value of \(y\) when \(x=0\).
  • \(b_1\) is a number called the slope. It is, on average, how much the value of \(y\) changes when the value of \(x\) increases by 1.

We will use software to find the values of \(b_0\) and \(b_1\).

However, we can roughly guess the values of the intercept by first drawing what looks like a sensible straight line through the data, and determining what that line predicts for the value of \(y\) when \(x=0\).

A rough guess of the slope can be made using the formula

\[ \text{slope} = \frac{\text{Change in $y$}}{\text{Corresponding change in $x$}} = \frac{\text{rise}}{\text{run}}. \] That is, a guess of the slope is the change in the value of \(y\) (the 'rise') divided by the corresponding change in the value of \(x\) (the 'run').

To demonstrate, consider the scatterplot in Fig. 36.1. I have drawn a sensible line on the graph to capture the relationship (your line may look a bit different). When \(x = 0\), the regression line predicts the value of \(y\) is about to be 2, so \(b_0\) is approximately 2.

To guess the slope, use the 'rise over run' idea. The animation below may help explain the rise-over-run idea. When the value of \(x\) increases from 1 to 5 (a change of \(5-1=4\)), the corresponding values of \(y\) change from 5 to 17 (a change of \(17 - 5 = 12\)). Then, use the formula:

\[\begin{align*} \frac{\text{rise}}{\text{run}} &= \frac{17 - 5}{5 - 1}\\ &= \frac{12}{4} = 3. \end{align*}\] The value of \(b_1\) is about \(3\). The regression line is approximately \(\hat{y} = 2 + (3\times x)\), usually written as

\[ \hat{y} = 2+3x. \]

The intercept has the same measurement units as the response variable. For example, with the red-deer data the intercept is measured in 'grams', the measurement units of the molar weight.

The measurement unit for the slope is the 'measurement units of the response variable', per 'measurement units of the explanatory variable'. For example, with the red-deer data the slope has the units of 'grams per year'.

An example scatterplot

FIGURE 36.1: An example scatterplot

You may like to play with the following interactive activity, which explores slopes and intercepts.

Example 36.2 (Estimating regression parameters) A study537 examined the number of cyclones \(y\) in the Australian region each year from 1969 to 2005, and the relationship with a climatological index called the Ocean Nino Index (ONI, \(x\)); see (Fig. 36.2),

When the value of \(x\) is zero, the predicted value of \(y\) is about 12; \(b_0\) is about 12. (You may get something slightly different.)

Notice that the intercept is the predicted value of \(y\) when \(x=0\), which is not at the left of the graph.

To guess the value of \(b_1\), use the 'rise over run' idea. When \(x\) is about \(-2\), the predicted value of \(y\) is about 17. When \(x\) is about \(2\), the predicted value of \(y\) is about 8.

So when the value of \(x\) changes by \(2 - (-2) = 4\), the value of \(y\) changes by \(8 - 17 = -9\) (a decrease of about 9). Hence, the value of \(b_1\) is approximately \(-9/4 = -2.25\). (You may get something slightly different.) Notice that the relationship has a negative direction, so the slope must be negative.

Using these guesses of \(b_0 = 12\) and \(b_1 = -2.25\), the regression line is approximately \[ \hat{y} = 12 - 2.25x. \]

The number of cyclones in the Australian region each year from 1969 to 2005, and the ONI for October, November, December

FIGURE 36.2: The number of cyclones in the Australian region each year from 1969 to 2005, and the ONI for October, November, December

In this section, we have seen how to understand a linear regression equation, and how an equation can be used to describe a fitted line. The above method gives a very crude guess of the values of the intercept \(b_0\) and the slope \(b_1\).

In practice, many reasonable lines could be drawn through a scatterplot of data. However, one of those lines is the 'best fitting line' in some sense538. Software calculates this 'line of best fit' for us.

36.3 Regression using software

In the population, the intercept is denoted by \(\beta_0\) and the slope is denoted by \(\beta_1\). These population values are unknown, and are estimated by the statistics \(b_0\) and \(b_1\) respectively.

The symbol \(\beta\) is the Greek letter 'beta', pronounced 'beater' (as in 'egg beater').
So \(\beta_0\) might be said as 'beater-zero', and \(\beta_1\) as 'beater-one'.

The formulas for computing \(b_0\) and \(b_1\) are ugly, so we will use software to do the calculations. As usual, the values of these population parameters are unknown, and the values of the sample statistics will change from sample to sample (so they have sampling variation).

For the red deer data again (Fig 34.2), part of the relevant output is shown in Fig. 36.3 (using jamovi) and Fig. 36.4 (using SPSS).

From the output, the slope \(b_1\) in the sample is \(b_1 = -0.181\), and the \(y\)-intercept \(b_0\) in the sample is \(b_0 = 4.398\). That is, the values of \(b_0\) and \(b_1\) are in the column labelled Estimate in jamovi, or the column labelled B in SPSS. These are the values of the two regression coefficients; then

\[ \hat{y} = 4.398 + (-0.181\times x), \] which is usually written more simply as

\[ \hat{y} = 4.398 - 0.181 x. \]

The sign of the slope \(b_1\) and the correlation coefficient \(r\) are always the same.

For example, if the slope is negative, the correlation coefficient will also be negative.

jamovi output for the red-deer data

FIGURE 36.3: jamovi output for the red-deer data

SPSS output for the red-deer data

FIGURE 36.4: SPSS output for the red-deer data

Example 36.3 (Regression coefficients) The regression equation for the cyclone data (Fig. 36.2) can be found from the jamovi output (Fig. 36.5): \[ \hat{y} = 12.14 - 2.23x, \] where \(x\) is the ONI (averaged over October, November, December) and \(y\) is the number of cyclones. These values are close the guesses made in Example 36.2.

jamovi output for the cyclone data

FIGURE 36.5: jamovi output for the cyclone data

You may like to play with the following interactive activity, which explores regression equations.

36.4 Regression for predictions

The regression equation for the red deer data

\[ \hat{y} = 4.398 - 0.181 x \] can be used to make predictions. For example, we could predict the average molar weight for deer \(10\) years old. Since \(x\) represents the age, use \(x=10\) in the regression equation:

\[\begin{eqnarray*} \hat{y} &=& 4.398 - (0.181\times 10)\\ &=& 4.398 - 1.81\\ &=& 2.588. \end{eqnarray*}\] Male red deer aged 10 years old are predicted to have a mean molar weight of 2.588 grams.

Some individual male red deer aged 10 will have molars weighing more than this, and some will have molars weighing less than this. The model predicts that the mean molar weight for male red deer aged 10 will be about 2.588 grams.

For male red deer \(12\) years of age, what is the predicted mean molar weight?

For male red deer \(20\) years of age, what is the predicted mean molar weight?

(Answer is here539.)

This last prediction may be a useful prediction... but it also may be rubbish.

The oldest deer in the data is aged 14.4 years, so the regression line may not even apply for deer aged over 14.4 years of age (red deer may not even live to 20 years of age). The prediction may be sensible... but it may not be either.

We don't know whether the prediction is sensible or not, because we have no data for deer aged over 14.4 years to inform us. Making prediction outside the range of the available data is called extrapolation, and extrapolation beyond the data can lead to nonsense predictions.

Definition 36.1 (Extrapolation) Extrapolation refers to making predictions outside the range of the available data. Extrapolation beyond the data can lead to nonsense predictions.

Extrapolating can lead to nonsense predictions.

36.5 Regression for understanding

The regression equation can be used to understand the relationship between the two variables. Consider again the red deer regression equation:

\[\begin{equation} \hat{y} = 4.398 - 0.181 x. \tag{36.1} \end{equation}\] What does it tell us about the relationship between \(x\) and \(y\)?

36.5.1 The meaning of \(b_0\)

\(b_0\) is the predicted value of \(y\) when \(x=0\). Equation (36.1) predicts a molar weight of \(4.398\) for a deer zero years of age, which is likely to be nonsense: it is extrapolating beyond the data (the youngest deer in the sample is aged 4.4 years).

The value of the intercept \(b_0\) is sometimes meaningful, but is often meaningless.

The value of the slope \(b_1\) is usually of greater interest, as it explains the relationship between the two variables.

36.5.2 The meaning of \(b_1\)

\(b_1\) tells us how much the value of \(y\) changes (on average) when the value of \(x\) increase by one.

For the red-deer data, \(b_1\) tells us how much the molar weight changes (on average) when age increases by one year.

Each extra year of age is associated with a change of \(-0.181\) grams in molar weight; that is, a decrease in molar weight by a mean of \(0.181\). The molars of some individual deer will lose more weight than this in some years, and some will lose less weight than this in some years... the value is a mean weight loss per year.

To demonstrate, for \(x=10\), \(y\) is predicted to be \(\hat{y}= 2.588\). For deer one year older than this (i.e. \(x=11\)) we predict \(y\) to be \(b_1 = -0.181\) higher (or, equivalently, 0.181 lower). That is, we would predict \(\hat{y}= 2.588 - 0.181 = 2.407\). (This is the same prediction made by using \(x=11\) in Eq. (36.1).)

If the value of \(b_1\) is positive, then the predicted values of \(y\) increase as the values of \(x\) increase.

If the value of \(b_1\) is negative, then the predicted values of \(y\) decrease as the values of \(x\) increase.

This interpretation of \(b_1\) explains the relationship: Each extra year of age reduces the weight of the molars by 0.181 grams, on average, in male red deer.

The units of measurement of the slope are the units of the response variable divided by the units of the explanatory variable (so in the deer example, the slope is \(-0.181\) grams per year).

Observe what happens if the slope is zero. Since \(b_1\) is the change in \(y\) (on average) when \(x\) increase by one, \(b_1 = 0\) means that the value of \(y\) changes by zero if the value of \(x\) changes by one.

In other words, if the value of \(x\) changes, the predicted value of \(y\) doesn't change. This is equivalent to saying that there is no relationship between the variables. (We would also find \(r=0\).)

If the value of the slope is zero, there is no linear relationship between \(x\) and \(y\).

In this case, the correlation coefficient is also zero.

36.6 Hypothesis testing

36.6.1 Introduction

The regression line is computed from the sample, assuming a linear relationship actually exists in the population. The (unknown) regression line in the population is

\[ \hat{y} = {\beta_0} + {\beta_1} x. \] From the sample, the estimate of the population regression line (Appendix C) is

\[ \hat{y} = {b_0} + {b_1} x. \] That is, the intercept in the population is \(\beta_0\) (estimated by \(b_0\)), and the slope in the population is \(\beta_1\) (estimated by \(b_1\)).

The sample can be used to ask questions about the population regression coefficients. As usual, the sample values can vary from sample to sample (and so have a sampling distribution).

Usually questions are asked about the slope, because the slope explains the relationship between the two variables (Sect. 36.5).

36.6.2 Hypotheses: Assumption

The null hypothesis is the usual 'no relationship' hypothesis. In this context, 'no relationship' means that the slope is zero (Sect. 36.5.2). Hence, the null hypotheses (about the population) is:

  • \(H_0\): \(\beta_1 = 0\).

This hypothesis proposes that \(b_1\) is not zero because of sampling variation. As part of the decision-making process, the null hypothesis is initially assumed to be true.

For the red deer data (Sect. 34.2), determining if a relationship exists between the age of the deer, and the weight of their molars, would test these hypotheses:

  • \(H_0\): \(\beta_1 = 0\);
  • \(H_1\): \(\beta_1 \ne 0\)

The parameter is \(\beta\), the population slope for the regression equation predicting molar weight from age.

The alternative hypothesis is two-tailed, based on the RQ.

36.6.3 Sampling distribution: Expectation

Assuming the null hypothesis is true (that \(\beta_1=0\)), we can describe what values the sample slope \(b_1\) are expected to take, through sampling variation.

The variation in the sample slope from sample to sample can be described (Fig. 36.6) using:

  • an approximate normal distribution,
  • with a mean of \(\beta_1 = 0\) (from \(H_0\)), and
  • a standard deviation, called the standard error of the slope, of \(\text{s.e.}(b_1)\).

The standard error is found using software (jamovi: Fig. 36.7; SPSS: Fig. 36.8).

The distribution of sample slope for the red deer data, if the population slope is 0

FIGURE 36.6: The distribution of sample slope for the red deer data, if the population slope is 0

36.6.4 The test statistic: Observation

The observed sample slope was \(b_1 = -0.181\). The test statistic would be found using the usual approach:

\[\begin{align*} t &= \frac{ b_1 - \beta_1}{\text{s.e.}(b_1)} \\ &= \frac{-0.181 - 0}{0.0289} = -6.27, \end{align*}\] where the values of \(b_1\) and \(\text{s.e.}(b_1)\) are taken from the software output. The \(t\)-score is also reported by the software.

36.6.5 \(P\)-value: Consistency with assumption

To determine if the statistic is consistent with the null hypothesis, the \(P\)-value can be approximated using the 68--95--99.7 rule, or taken from software output (jamovi: Fig. 36.7; SPSS: Fig. 36.8).

Using software, the \(P\)-value is \(P<0.001\).

We write:

The sample presents very strong evidence (\(t = -6.27\); one-tailed \(P<0.001\)) that the slope in the population between age of the deer and molar weight is not zero (slope: \(-0.181\)).

jamovi output for the red-deer data

FIGURE 36.7: jamovi output for the red-deer data

SPSS output for the red-deer data

FIGURE 36.8: SPSS output for the red-deer data

Example 36.4 (Emergency department patients) A study examined the relationship between the number of emergency department (ED) patients and the number of days following the distribution of monthly welfare monies540 from 1986 to 1988 in Minneapolis, USA.

The data (extracted from Fig. 2 of Brunette, Kominsky, and Ruiz541) is plotted in Fig. 36.9, and the jamovi analysis shown in Fig. 36.10.

The regression line is estimated as

\[ \hat{y} = 150.19 - 0.348x, \] where \(y\) represents the mean number of ED patients, and \(x\) the number of days since welfare distribution.

This regression equation suggests that each extra day after welfare distribution is associated with a decrease in the number of ED patients of about 0.35. It may be easier to understand this way:

Each 10 extra days after welfare distribution is associated with a decrease in the number of ED patients of about \(10\times 0.35 = 3.5\).

The scatterplot and the regression equation suggests a negative relationship between the number of ED patients and the days after distribution. However, we know that every sample is likely to be different, so the relationship may not actually be present in the population. So we test these hypotheses:

  • \(H_0\): \(\beta_1 = 0\) where \(\beta_1\) is the population slope
  • \(H_1\): \(\beta_1 \ne 0\) (i.e., two-tailed, based on the authors' aim)

The output shows that the test statistic is \(t = -7.45\), which is very large; unsurprisingly, the two-tailed \(P\)-value is very small: \(P<0.001\).

We write:

There is very strong evidence (\(t = -7.45\); two-tailed \(P < 0.001\)) of a relationship between the mean number of ED patients and the number of days after welfare distribution (slope: \(-0.348\)).

Scatterplot of the number of emergency department patients, and the number of days since distribution of welfare

FIGURE 36.9: Scatterplot of the number of emergency department patients, and the number of days since distribution of welfare

jamovi output for the emergency department data

FIGURE 36.10: jamovi output for the emergency department data

36.7 Confidence intervals

Reporting the CI for the slope is also useful, which can be obtained from software or computed manually.

Using the output (jamovi: Fig. 36.7; SPSS: Fig. 36.8), what is the approximate 95% CI for \(\beta_1\)?

CIs have the form \[ \text{statistic} \pm ( \text{multiplier} \times \text{standard error}), \] The multiplier is two for an approximate 95% CI, so (using the standard error reported by the software), we obtain \(-0.181 \pm (2\times 0.029)\), or \(-0.181 \pm 0.058\), or from \(-0.239\) to \(-0.123\).

Software can be asked to produce exact CIs too (jamovi: Fig. 36.11; SPSS: Fig. 36.12). The approximate and exact 95% CIs are very similar.

jamovi output for the red-deer data, including the CIs

FIGURE 36.11: jamovi output for the red-deer data, including the CIs

SPSS output for the red-deer data, including the CIs

FIGURE 36.12: SPSS output for the red-deer data, including the CIs

We write:

The sample presents very strong evidence (\(P < 0.001\); \(t = -6.275\)) of a relationship between age and the weight of molars in male red deer (slope: \(-0.181\); \(n= 78\); 95% CI from \(-0.239\) to \(-0.124\)) in the population.

Example 36.5 (Emergency department patients) In Example 36.4, the jamovi output does not give the 95% CI for the slope.

However, since CIs have the form \[ \text{statistic} \pm ( \text{multiplier} \times \text{standard error}), \] the CI is easily computed: \[ -0.34790 \pm (2\times 0.046672), \]
or \(-0.34790 \pm 0.093344\). This is equivalent to \(-0.441\) to \(-0.255\).

Combining with information from Example 36.4, we write:

The sample presents very strong evidence (\(P < 0.001\); \(t = -7.45\)) of a relationship between the mean number of ED patients and the numbers of days after welfare distribution (slope: \(-0.348\); \(n = 30\); 95% CI from \(-0.441\) to \(-0.255\)) in the population.

36.8 Statistical validity conditions

As usual, these results hold under certain conditions to be met. The conditions for which the test is statistically valid are the same as for correlation (Sect. 35.4.3):

  1. The relationship is approximately linear.
  2. The variation in the response variable is approximately constant for all values of the explanatory variable.
  3. The sample size is at least 25.

The sample size of 25 is a rough figure here, and some books give other values.

In addition to the statistical validity condition, the test will be

Example 36.6 (Statistical validity) For the red deer data, the relationship is approximately linear, and the variation in molar weight appears to be somewhat constant for various ages (Fig. 34.2), so regression is appropriate. The sample size is \(n=78\). The results from the hypothesis test should be statistically valid.

Are the conditions for statistically validity met for the cyclones data (Fig. 36.2)?

Example 36.7 (Emergency department patients) In Example 36.4, the scatterplot (Fig. 36.9) shows that the relationship is approximately linear, that the variation in ED patients seems reasonably constant for different numbers of days after distribution, and the sample size is larger than 30.

The test (Example. 36.4) and the CI (Example. 36.5) should be statistically valid.

36.9 Example: Obstructive sleep apnoea

In a study of obstructive sleep apnoea (OSA) in adults with Down Syndrome,542 \(n = 60\) adults underwent a sleep study and had various data recorded.

The main response variable of interest was OSA severity (measured using the Respiratory Event Index, REI). REI means the average number of episodes of sleep disruption (according to specific criteria) per hour of sleep.

One research question is

Among Down Syndrome adults, is there a relationship between the REI and neck size?

Here, \(x\) is the neck size (in cm), and \(y\) is the REI value.

The data are shown in Fig. 36.13.

FIGURE 36.13: The Obstructive sleep apnoea data set

Scatterplot of the neck circumference vs REI for Down Syndrome adults

FIGURE 36.14: Scatterplot of the neck circumference vs REI for Down Syndrome adults

Using the jamovi output (Fig. 36.15) the value of the slope and \(y\)-intercept in the sample are \(b_0 = -0.193\) and \(b_1 = 0.047\).

jamovi output for the Obstructive sleep apnoea data

FIGURE 36.15: jamovi output for the Obstructive sleep apnoea data

The values of the slope and \(y\)-intercept in the sample are \(b_0 = -24.47\) and \(b_1 = 1.36\). The regression equation is

\[ \hat{y} = -24.47 + 1.36x. \]

The slope means that for each one centimetre increase in neck circumference, the number of sleep disruptions per hour increase (on average) by about 1.36.

Each sample will produce slightly different sample slopes, so we can test to see if the slope in the population is non-zero due to sampling variation, using a hypothesis test:

  • \(H_0\): \(\beta_1 = 0\);
  • \(H_1\): \(\beta_1 \ne 0\) (that is, two-tailed).

The parameter is \(\beta_1\). From the software output, \(t = 2.09\) and the two-tailed \(P\)-value is \(P = 0.041\). This means there moderate evidence that the neck circumference is associated with greater REI.

The approximate 95% CI for the population slope \(\beta_1\) is

\[ 1.3663 \pm (2\times 0.65441), \] or from 0.057 to 2.68.

From the scatterplot (Fig. 36.14) the results appear statistically valid. We write:

The sample presents moderate evidence (\(t = 2.09\); two-tailed \(P = 0.041\)) of a relationship between neck circumference and REI (slope: \(1.36\); \(n = 60\); 95% CI from \(0.057\) to \(2.68\)) in the population.

36.10 Example: Food digestibility

A study evaluated various food for sheep.543 One combination of variables assessed is shown in Fig. 34.6.

The RQ is:

Does the digestible energy requirement of feed increase with dry matter digestibility percentage (and if so, what is the relationship)?

In this study, \(x\) is the dry matter weight digestibility percentage, and \(y\) is the digestible energy. The data are shown in Fig. 36.16. Using the software output (Fig. 36.17 (jamovi); Fig. 36.18 (SPSS)), the values of the slope and \(y\)-intercept in the sample are \(b_0 = -0.193\) and \(b_1 = 0.047\). The regression equation is

\[ \hat{y} = -0.193 + 0.047x. \]

FIGURE 36.16: The digestibility data set

The slope means that when the dry matter weight digestibility increases by 1 percentage point, the digestible energy increases, on average, by 0.047 Cal/gram.

Each sample will produce slightly different sample slopes, so we can test to see if the slope in the population is non-zero due to sampling variation, using a hypothesis test:

  • \(H_0\): \(\beta_1 = 0\);
  • \(H_1\): \(\beta_1 > 0\).

The parameter is \(\beta_1\), the population slope for the regression equation predicting digestible energy from dry matter weight.

The alternative hypothesis is one-tailed, based on the RQ.

From the software output, \(t = 39.322\), which is huge; the two-tailed \(P\)-value is \(P<0.001\). Since we have a one-tailed alternative hypothesis, the \(P\)-value is less than \(0.001/2 = 0.0005\). There is very strong evidence that the digestible energy increases as the dry matter weight digestibility increases.

The approximate 95% CI for the population slope \(\beta_1\) is

\[ 0.047 \pm (2\times 0.001), \] or from 0.045 to 0.049.

jamovi output for the sheep-feed data

FIGURE 36.17: jamovi output for the sheep-feed data

SPSS output for the sheep-feed data

FIGURE 36.18: SPSS output for the sheep-feed data

The results are statistically valid. We write:

The sample presents very strong evidence (\(t = 39.322\); one-tailed \(P < 0.0005\)) of a relationship between dry matter weight digestibility and the digestible energy (slope: \(0.047\); \(n = 36\); 95% CI from \(-0.045\) to \(-0.049\)) in the population.

Click on the hotspots in the following image, to see what the areas under the normal curve mean.

36.11 Summary

In this chapter, we have learnt about regression, which mathematically describes the relationship between two quantitative variables. The response variable is denoted by \(y\), and the explanatory variable by \(x\). The linear relationship between them (the regression equation), in the sample, is

\[ \hat{y} = b_0 + b_1 x, \] where \(b_0\) is a number (the intercept), \(b_1\) is a number (the slope), and the 'hat' above the \(y\) indicates that the equation gives an predicted mean value of \(y\) for the given \(x\) value.

The intercept is the predicted mean value of \(y\) when the value of \(x\) is zero. The slope is how much the predicted mean value of \(y\) changes, on average, when the value of \(x\) increases by 1.

The regression equation can be used to make predictions or to understand the relationship between the two variables. Predictions made with values of \(x\) outside the values of \(x\) used to create the regression equation (called extrapolation) may not be reliable.

In the population, the regression equation is

\[ \hat{y} = \beta_0 + \beta_1 x. \] To test a hypothesis about a population slope \(\beta_1\), based on the value of the sample slope \(b_1\), assume the value of \(\beta_1\) in the null hypothesis (usually zero) to be true. Then, the sample slope varies from sample to sample and, under certain statistical validity conditions, varies with an approximate normal distribution centered around the hypothesised value of \(\beta_1\), with a standard deviation of \(\text{s.e.}(b_1)\). This distribution describes what values of the sample slope could be expected in the sample if the value of \(\beta_1\) in the null hypothesis was true. The test statistic is

\[ t = \frac{ b_1 - \beta_1}{\text{s.e.}(b_1)}, \] where \(\beta_1\) is the hypothesised value given in the null hypothesis (usually zero). The \(t\)-value is like a \(z\)-score, and so an approximate \(P\)-value can be estimated using the 68--95--99.7 rule.

The following short video may help explain some of these concepts:

36.12 Quick review questions

A study of athletes544 examined the relationship between the height and weight of \(n=37\) rowers at the Australian Institute of Sport (AIS), as shown in Fig. 36.19.

Scatterplot of Weight against Height rowers at the AIS

FIGURE 36.19: Scatterplot of Weight against Height rowers at the AIS

  1. Using the 'rise over run' idea, the slope is approximately

  2. The \(y\)-variable is

  3. The regression equation is \(\hat{y} = -138 + 1.2 x\). What does \(x\) represent?

  4. What does the 'hat' above the \(y\) mean?

  5. To two decimal places, what weight would be predicted for a rower who is 180cm tall?

  6. The standard error of the slope is 0.112. What is the value of the test statistic (to one decimal place) to test if the population slope is zero?

  7. True or false? The \(P\)-value for this test will be very small

  8. True or false? The units of the slope are kg/cm

  9. True or false? Making a prediction for the weight of a rower weighing 220 kg would be an example of extrapolation

Progress:

36.13 Exercises

Selected answers are available in Sect. D.33.

Exercise 36.1 In wastewater treatment facilities, air from biofiltration is passed through a membrane and dissolved in water, and is transformed into harmless byproducts.545 The removal efficiency \(y\) (in %) may depend on the inlet temperature (in \(^\circ\)C; \(x\)). The RQ is

In treating biofiltation wastewater, how does the removal efficiency depend on the inlet temperature?

Using the scatterplot (Fig. 36.20) and software output (Fig. 36.21 (jamovi); Fig. 36.22 (SPSS)), answer these questions.

  1. Write down the value of the slope (\(b_1\)) and \(y\)-intercept (\(b_0\)).
  2. Write down the regression equation.
  3. Interpret the slope (\(b_1\)).
  4. Test for a relationship in the population, by first writing the hypotheses.
  5. Write down the \(t\)-score and \(P\)-value from the software output.
  6. Determine an approximate 95% CI for the population slope \(\beta_1\).
  7. Write a conclusion.
The relationship between removal efficiency and inlet temperature

FIGURE 36.20: The relationship between removal efficiency and inlet temperature

jamovi regression output for the removal-efficiency data

FIGURE 36.21: jamovi regression output for the removal-efficiency data

SPSS regression output for the removal-efficiency data

FIGURE 36.22: SPSS regression output for the removal-efficiency data

Exercise 36.2 A study (Myers,546 p. 75) of American footballers measured the right-leg strengths \(x\) of 13 players (using a weight lifting test), and the distance \(y\) they punt a football (with their right leg).

  1. Use the plot (Fig. 36.23) to guess of the values of the intercept and slope.
  2. Using the jamovi output (Fig. 36.24), write down the value of the slope (\(b_1\)) and \(y\)-intercept (\(b_0\)).
  3. Hence write down the regression equation.
  4. Interpret the slope (\(b_1\)).
  5. Write the hypotheses for testing for a relationship in the population
  6. Write down the \(t\)-score and \(P\)-value from the output.
  7. Determine an approximate 95% CI for the population slope \(\beta_1\).
  8. Write a conclusion.
Punting distance and right leg strength

FIGURE 36.23: Punting distance and right leg strength

jamovi regression output for the punting data

FIGURE 36.24: jamovi regression output for the punting data

Exercise 36.3 A study547 of gas engines measured the throttle angle (\(x\)) and the manifold air pressure (\(y\)) as a fraction of the maximum value.

  1. The value of \(r\) is given in the paper as \(0.972986604\). Comment on this value, and what it means.
  2. Comment on the use of a regression model, based on the scatterplot (Fig. 36.25).
  3. The authors fitted the following regression model: \(y = 0.009 + 0.458x\). Identify errors that the researchers have made when giving this regression equation (there are more than one).
  4. Critique the researchers' approach.
Manifold air pressure plotted against throttle angle for an internal-combustion gas engine

FIGURE 36.25: Manifold air pressure plotted against throttle angle for an internal-combustion gas engine

Exercise 36.4 A study of hot mix asphalt548 created \(n=42\) samples of asphalt and measured the volume of air voids and the bitumen content by weight (Fig. 36.26). Use the software output (Fig. 36.27) to answer these questions.

  1. Write down the regression equation.
  2. Interpret what the regression equation means.
  3. Perform a test to determine is there is a relationship between the variables.
  4. Predict the mean percentage of air voids by volume when the percentage bitumen is 5.0%. Do you expect this to be a good prediction? Why or why not?
  5. Predict the mean percentage of air voids by volume when the percentage bitumen is 6.0%. Do you expect this to be a good prediction? Why or why not?
Air voids in bitumen samples

FIGURE 36.26: Air voids in bitumen samples

jamovi regression output for the bitumen data

FIGURE 36.27: jamovi regression output for the bitumen data

Exercise 36.5 A study of \(n=31\) people on the use of sunscreen549 explored the relationship between the time (in minutes) spent on sunscreen application \(x\) and the amount (in grams) of sunscreen applied (\(y\)). The fitted regression equation was \(\hat{y} = 0.27 + 2.21x\).

  1. Interpret the meaning of \(b_0\) and \(b_1\). Do they seems sensible?
  2. According to the article, a hypothesis for testing \(\beta_0\) produced a \(P\)-value much larger than \(0.05\). What does this mean?
  3. If someone spent 8 minutes applying sunscreen, how much sunscreen would you predict that they used?
  4. The article reports that \(R^2=0.64\). Interpret this value.
  5. What is the value of the correlation coefficient?

Exercise 36.6 One study550 stated:

In developing countries [...] logistic problems prevent the weighing of every newborn child. A study was performed to see whether other simpler measurements could be substituted for weight to identify neonates of low birth weight and those at risk.

--- Bhargava et al.551 p. 1617

The relationship between infant chest circumference (in cm) \(x\) and birth weight (in grams) \(y\) was given as:

\[ \hat{y} = -3440.2403 + 199.2987x. \] The correlation coefficient was \(r = 0.8696\) with \(P < 0.001\).

  1. Based on the regression equation only, could chest circumference be used as a useful predictor of birth weight? Explain.
  2. Based on the correlation information only, could chest circumference be used as a useful predictor of birth weight? Explain.
  3. Interpret the intercept and the slope of the regression equation.
  4. What units of measurement are the intercept and slope measured in?
  5. Predict the birth weight of an infant with a chest circumference of 30cm.
  6. Critique the way in which the regression equation and correlation coefficient are reported.