Prediction interval (PI) for Y given x

The quantity of interest here is a future observation of $Y$, $Y_f$ say, when $x$ takes the value $x_f$, which denotes the value of the explanatory variable at the position where a prediction of $Y$ is required. Note that the observation $(Y_f, x_f)$ was not used to construct the regression model or estimate regression parameters. The expected value $E(Y|x_f)$ can be written in the form $\mathbf{b}_f^T\boldsymbol{\beta}$ . For example, with a simple linear regression, $E(Y) = \alpha+\beta x$ , we can write $E(Y|x_f)=\alpha+\beta x_f = \mathbf{b}^T\boldsymbol{\beta}$ , where $\mathbf{b}^T = (1, x_f)$ and $\boldsymbol{\beta}^T = (\alpha,\beta)$.

A prediction interval for $y_f$ is

$$\mathbf{b}^T\boldsymbol{\hat{\beta}}\pm t\left(n-p; \frac{1+c}{2}\right)\sqrt{\frac{RSS}{n-p}(1+\mathbf{b}^T(\mathbf{X}^T\mathbf{X})^{-1}\mathbf{b})}.$$

Simple linear regression

Construct a 95% C.I. (confidence interval) of $\beta$ in the model $y_i = \alpha+\beta x_{1i}+\gamma x_{2i}+\epsilon_i$ . We can write this C.I. as

$$\hat{\beta} \pm t(n-p; 0.975)\mathrm{s.e.}(\hat{\beta})$$

where

$$\mathrm{s.e.}(\hat{\beta}) = \sqrt{\frac{RSS}{n-p}\mathbf{b}^T(\mathbf{X}^T\mathbf{X})^{-1}\mathbf{b}}$$

and

$$\mathbf{b} = \left( \begin{array}{c} 0 \\ 1 \\ 0\\ \end{array} \right)$$

R automatically prints the standard error (s.e.) of each individual parameter when it fits a regression model. This makes the construction of C.I.s for each parameter very easy given these estimates.

A parameter estimate is a random variable, since it can take several values, and in probability terms the estimated standard error of a parameter estimate is an estimate of its standard deviation.