Interval estimate for \(\mathbf{b}^T\boldsymbol{\beta}\)

In order to construct an interval estimate, we again apply the ideas already discussed in the previous lecture. In particular, an interval estimate for \(\mathbf{b}^T\boldsymbol{\beta}\) with confidence \(c\) is

\[\mathbf{b}^T\boldsymbol{\hat{\beta}}\pm t\left(n-p; \frac{1+c}{2}\right)\sqrt{\frac{RSS}{n-p}(\mathbf{b}^T(\mathbf{X}^T\mathbf{X})^{-1}\mathbf{b})}.\]

For information only

For some level of confidence \(c\), we essentially are looking to find the value of \(\delta\) such that

\[\begin{eqnarray*} P(|\mathbf{b}^T\boldsymbol{\hat{\beta}}-\mathbf{b}^T\boldsymbol{\beta}| \leq \delta ) &=& c\\ \end{eqnarray*}\]

In words, this means we want the probability that our estimated value \(\mathbf{b}^T\boldsymbol{\hat{\beta}}\) lies within \(\delta\) of the true value \(\mathbf{b}^T\boldsymbol{\beta}\) to be \(c\). We would typically want \(c\) to be close to 1 (i.e., we want to be confident!).

\[\begin{eqnarray*} P\bigg(\bigg|\frac{(\mathbf{b}^T\boldsymbol{\hat{\beta}}-\mathbf{b}^T\boldsymbol{\beta})}{\sqrt{\frac{RSS}{n-p}\mathbf{b}^T(\mathbf{X}^T\mathbf{X})^{-1}\mathbf{b}}}\bigg| \leq \frac{\delta}{\sqrt{\frac{RSS}{n-p}\mathbf{b}^T(\mathbf{X}^T\mathbf{X})^{-1}\mathbf{b}}}\bigg) &=& c\\ \end{eqnarray*}\]

In the plot above, showing a t-distribution, we want the area enclosed by the solid black vertical lines to contain \(100c\)% of the distribution. We want to therefore know the value (\(t\)) at which the solid black vertical lines intersect the x-axis.

\[\begin{eqnarray*} t&=&\frac{\delta}{\sqrt{\frac{RSS}{n-p}\mathbf{b}^T(\mathbf{X}^T\mathbf{X})^{-1}\mathbf{b}}}\\ \delta &=& t \times \sqrt{\frac{RSS}{n-p}\mathbf{b}^T(\mathbf{X}^T\mathbf{X})^{-1}\mathbf{b}} \end{eqnarray*}\]

Given the symmetry of the t-distribution, then \(t=t(n-p,(1+c)/2)\).