Interval estimate for $\mathbf{b}^T\boldsymbol{\beta}$

In order to construct an interval estimate, we again apply the ideas already discussed in the previous lecture. In particular, an interval estimate for $\mathbf{b}^T\boldsymbol{\beta}$ with confidence $c$ is

$$\mathbf{b}^T\boldsymbol{\hat{\beta}}\pm t\left(n-p; \frac{1+c}{2}\right)\sqrt{\frac{RSS}{n-p}(\mathbf{b}^T(\mathbf{X}^T\mathbf{X})^{-1}\mathbf{b})}.$$

For information only

For some level of confidence $c$, we essentially are looking to find the value of $\delta$ such that

$$\begin{eqnarray*} P(|\mathbf{b}^T\boldsymbol{\hat{\beta}}-\mathbf{b}^T\boldsymbol{\beta}| \leq \delta ) &=& c\\ \end{eqnarray*}$$

In words, this means we want the probability that our estimated value $\mathbf{b}^T\boldsymbol{\hat{\beta}}$ lies within $\delta$ of the true value $\mathbf{b}^T\boldsymbol{\beta}$ to be $c$. We would typically want $c$ to be close to 1 (i.e., we want to be confident!).

$$\begin{eqnarray*} P\bigg(\bigg|\frac{(\mathbf{b}^T\boldsymbol{\hat{\beta}}-\mathbf{b}^T\boldsymbol{\beta})}{\sqrt{\frac{RSS}{n-p}\mathbf{b}^T(\mathbf{X}^T\mathbf{X})^{-1}\mathbf{b}}}\bigg| \leq \frac{\delta}{\sqrt{\frac{RSS}{n-p}\mathbf{b}^T(\mathbf{X}^T\mathbf{X})^{-1}\mathbf{b}}}\bigg) &=& c\\ \end{eqnarray*}$$

In the plot above, showing a t-distribution, we want the area enclosed by the solid black vertical lines to contain $100c$% of the distribution. We want to therefore know the value ($t$) at which the solid black vertical lines intersect the x-axis.

$$\begin{eqnarray*} t&=&\frac{\delta}{\sqrt{\frac{RSS}{n-p}\mathbf{b}^T(\mathbf{X}^T\mathbf{X})^{-1}\mathbf{b}}}\\ \delta &=& t \times \sqrt{\frac{RSS}{n-p}\mathbf{b}^T(\mathbf{X}^T\mathbf{X})^{-1}\mathbf{b}} \end{eqnarray*}$$

Given the symmetry of the t-distribution, then $t=t(n-p,(1+c)/2)$.