Prediction interval (PI) for Y given x

The quantity of interest here is a future observation of \(Y\), \(Y_f\) say, when \(x\) takes the value \(x_f\), which denotes the value of the explanatory variable at the position where a prediction of \(Y\) is required. Note that the observation \((Y_f, x_f)\) was not used to construct the regression model or estimate regression parameters. The expected value \(E(Y|x_f)\) can be written in the form \(\mathbf{b}_f^T\boldsymbol{\beta}\) . For example, with a simple linear regression, \(E(Y) = \alpha+\beta x\) , we can write \(E(Y|x_f)=\alpha+\beta x_f = \mathbf{b}^T\boldsymbol{\beta}\) , where \(\mathbf{b}^T = (1, x_f)\) and \(\boldsymbol{\beta}^T = (\alpha,\beta)\).

A prediction interval for \(y_f\) is

\[\mathbf{b}^T\boldsymbol{\hat{\beta}}\pm t\left(n-p; \frac{1+c}{2}\right)\sqrt{\frac{RSS}{n-p}(1+\mathbf{b}^T(\mathbf{X}^T\mathbf{X})^{-1}\mathbf{b})}.\]

Simple linear regression

Construct a 95% C.I. (confidence interval) of \(\beta\) in the model \(y_i = \alpha+\beta x_{1i}+\gamma x_{2i}+\epsilon_i\) . We can write this C.I. as

\[\hat{\beta} \pm t(n-p; 0.975)\mathrm{s.e.}(\hat{\beta})\]

where

\[\mathrm{s.e.}(\hat{\beta}) = \sqrt{\frac{RSS}{n-p}\mathbf{b}^T(\mathbf{X}^T\mathbf{X})^{-1}\mathbf{b}}\]

and

\[\mathbf{b} = \left( \begin{array}{c} 0 \\ 1 \\ 0\\ \end{array} \right)\]

R automatically prints the standard error (s.e.) of each individual parameter when it fits a regression model. This makes the construction of C.I.s for each parameter very easy given these estimates.

A parameter estimate is a random variable, since it can take several values, and in probability terms the estimated standard error of a parameter estimate is an estimate of its standard deviation.