5.4 Asymptotic confidence intervals

We assume now that the rv \(X\) that represents the population follows a distribution that belongs to a parametric family of distributions \(\{F(\cdot;\theta):\theta\in\Theta\},\) and that we want to obtain a confidence interval for \(\theta.\) This family does not have to be normal and may be even unknown. In this situation, we may have an asymptotic pivot of the form

\[\begin{align} Z(\theta)=\frac{\hat{\theta}-\theta}{\hat{\sigma}(\hat{\theta})}\stackrel{d}{\longrightarrow}\mathcal{N}(0,1).\tag{5.6} \end{align}\]

Then, an asymptotic confidence interval for \(\theta\) is given by

\[\begin{align*} \mathrm{ACI}_{1-\alpha}(\theta):=\left[\hat{\theta}\mp z_{\alpha/2}\hat{\sigma}(\hat{\theta})\right]. \end{align*}\]

This confidence interval has a confidence level approximate to \(1-\alpha,\) that is

\[\begin{align*} \mathbb{P}(\theta\in\mathrm{ACI}_{1-\alpha}(\theta))\approx 1-\alpha. \end{align*}\]

This approximation becomes more accurate as the sample size \(n\) grows due to (5.6).

5.4.1 Asymptotic confidence interval for the mean

Let \(X\) be a rv with \(\mathbb{E}[X]=\mu\) and \(\mathbb{V}\mathrm{ar}[X]=\sigma^2,\) both being unknown parameters, and let \((X_1,\ldots,X_n)\) be a srs of \(X.\) In Example 3.17 we have seen that

\[\begin{align*} Z(\mu)=\frac{\bar{X}-\mu}{S'/\sqrt{n}}\stackrel{d}{\longrightarrow} \mathcal{N}(0,1). \end{align*}\]

Therefore, an asymptotic confidence interval for \(\mu\) at the confidence level \(1-\alpha\) is

\[\begin{align*} \mathrm{ACI}_{1-\alpha}(\mu)=\left[\bar{X}\mp z_{\alpha/2}\frac{S'}{\sqrt{n}}\right]. \end{align*}\]

Example 5.8 The shopping times of \(n=64\) random customers at a local supermarket are measured. The sample mean and the quasivariance were \(33\) and \(256\) minutes, respectively. Estimate the average shopping time, \(\mu,\) of a customer with a confidence level \(0.90.\)

The asymptotic confidence interval for \(\mu\) at significance level \(\alpha=0.10\) is

\[\begin{align*} \mathrm{ACI}_{0.90}(\mu)=\left[\bar{X}\mp z_{0.05}\frac{S'}{\sqrt{n}}\right]\approx\left[33\mp 1.645\sqrt{\frac{256}{64}}\right]\approx[29.71,36.29]. \end{align*}\]

Example 5.9 Let \((X_1,\ldots,X_n)\) be a srs of a rv with distribution \(\mathrm{Pois}(\lambda).\) Let us compute an asymptotic confidence interval at significance level \(\alpha\) for \(\lambda.\)

In Exercise 3.11 we have seen that

\[\begin{align*} \frac{\bar{X}-\lambda}{\sqrt{\bar{X}/n}}\stackrel{d}{\longrightarrow}\mathcal{N}(0,1). \end{align*}\]

Therefore, the asymptotic confidence interval for \(\lambda\) is

\[\begin{align*} \mathrm{ACI}_{1-\alpha}(\lambda)=\left[\bar{X}\mp z_{\alpha/2} \sqrt{\frac{\bar{X}}{n}}\right]. \end{align*}\]

5.4.2 Asymptotic confidence interval for the proportion

Let \(X_1,\ldots,X_n\) be iid rv’s with \(\mathrm{Ber}(p)\) distribution. We are going to obtain an asymptotic confidence interval at confidence level \(1-\alpha\) for \(p.\)

In Example 3.18 we have seen that

\[\begin{align*} Z(p)=\frac{\hat{p}-p}{\sqrt{\hat{p}(1-\hat{p})/n}}\stackrel{d}{\longrightarrow}\mathcal{N}(0,1). \end{align*}\]

Therefore, an asymptotic confidence interval for \(p\) is

\[\begin{align*} \mathrm{ACI}_{1-\alpha}(p)=\left[\hat{p}\mp z_{\alpha/2}\sqrt{\frac{\hat{p} (1-\hat{p})}{n}}\right]. \end{align*}\]

This asymptotic confidence interval can be easily extended to the two proportions case.

Example 5.10 Two brands of refrigerators, A and B, have a warranty of one year. In a random sample of \(n_A=50\) refrigerators of A, \(12\) failed before the end of the warranty period. From the random sample of \(n_B=60\) refrigerators of B, \(12\) failed before the expiration of the warranty. Estimate the difference of the failure proportions during the warranty period at the confidence level \(0.98.\)

Let \(p_A\) and \(p_B\) be the proportion of failures for A and B, respectively. By the CLT we know that, for a large sample size, the sample proportions \(\hat{p}_A\) and \(\hat{p}_B\) verify

\[\begin{align*} \hat{p}_A\cong\mathcal{N}\left(p_A, \frac{p_A (1-p_A)}{n_A}\right),\quad \hat{p}_B\cong\mathcal{N}\left(p_B,\frac{p_B (1-p_B)}{n_B}\right). \end{align*}\]

Then, for large \(n_A\) and \(n_B\) it is verified

\[\begin{align*} \hat{p}_A-\hat{p}_B\cong\mathcal{N}\left(p_A-p_B,\frac{p_A (1-p_A)}{n_A}+\frac{p_B (1-p_B)}{n_B}\right). \end{align*}\]

Applying Corollary 3.3 and Theorem 3.4, a pivot is

\[\begin{align*} Z(p_A-p_B)=\frac{(\hat{p}_A-\hat{p}_B)-(p_A-p_B)}{\sqrt{{\frac{\hat{p}_A (1-\hat{p}_A)}{n_A}+\frac{\hat{p}_B (1-\hat{p}_B)}{n_B}}}}\stackrel{d}{\longrightarrow} \mathcal{N}(0,1) \end{align*}\]

so an asymptotic confidence interval for \(p_A-p_B\) is

\[\begin{align*} \mathrm{ACI}_{0.98}(p_A-p_B)=\left[(\hat{p}_A-\hat{p}_B)\mp z_{0.01}\sqrt{\frac{\hat{p}_A (1-\hat{p}_A)}{n_A}+\frac{\hat{p}_B (1-\hat{p}_B)}{n_B}}\right]. \end{align*}\]

The sample proportions for the refrigerators are \(\hat{p}_A=12/50=0.24\) and \(\hat{p}_B=12/60=0.20,\) so the above asymptotic confidence interval is

\[\begin{align*} \mathrm{ACI}_{0.98}(p_A-p_B)&\approx\left[(0.24-0.20)\mp 2.33\sqrt{\frac{0.24\times 0.76}{50}+\frac{0.20\times 0.80}{60}}\right]\\ &\approx[-0.145,0.225]. \end{align*}\]

This confidence interval contains zero. This indicates that \(p_A=p_B\) with a confidence of \(0.98,\) that is, that the probabilities of failure of the refrigerators of the two brands are not different.

5.4.3 Asymptotic maximum likelihood confidence intervals

Theorem 4.1 and Corollary 4.2 provide a readily usable asymptotic pivot for \(\theta\) based on the very general maximum likelihood estimator:

\[\begin{align*} Z(\theta)=\frac{\hat{\theta}_{\mathrm{MLE}}-\theta}{1\big/\sqrt{n\mathcal{I}(\hat{\theta}_{\mathrm{MLE}})}}=\sqrt{n\mathcal{I}(\hat{\theta}_{\mathrm{MLE}})}\left(\hat{\theta}_{\mathrm{MLE}}-\theta\right)\stackrel{d}{\longrightarrow}\mathcal{N}(0,1). \end{align*}\]

The result can be precised as follows.

Theorem 5.1 (Asymptotic maximum likelihood confidence intervals) Let \(X\sim f(\cdot;\theta).\) Under the conditions of Theorem 4.1 and Corollary 4.2, the asymptotic maximum likelihood confidence interval

\[\begin{align} \mathrm{ACI}_{1-\alpha}(\theta)=\left[\hat{\theta}_{\mathrm{MLE}}\mp \frac{z_{\alpha/2}}{\sqrt{n\mathcal{I}(\hat{\theta}_{\mathrm{MLE}})}}\right] \tag{5.7} \end{align}\]

is such that \(\mathbb{P}(\theta \in \mathrm{ACI}_{1-\alpha}(\theta))\to 1 - \alpha\) as \(n\to\infty.\)

Remark. An analogous result can be established for a discrete rv \(X\sim F(\cdot;\theta).\)

Remark. Due to Corollary 4.3, \(\mathcal{I}(\hat{\theta}_{\mathrm{MLE}})\) can be replaced with \(\hat{\mathcal{I}}(\hat{\theta}_{\mathrm{MLE}})\) in (5.7) without altering the asymptotic coverage probability. This is especially useful when \(\mathcal{I}(\theta)\) is difficult to compute, since \(\hat{\mathcal{I}}(\theta)=\frac{1}{n}\sum_{i=1}^n (\partial\log f(X_i;\theta)/\partial\theta)^2\) is straightforward to compute from the srs \((X_1,\ldots,X_n)\) from \(X\sim f(\cdot;\theta).\)

Example 5.11 Let us compute the asymptotic maximum likelihood confidence interval for \(\lambda\) in a \(\mathrm{Exp}(\lambda)\) distribution.

We know from Example 4.12 that \(\hat{\lambda}_{\mathrm{MLE}}=1/{\bar{X}}\) and \(\mathcal{I}(\lambda)=1/\lambda^2.\) Therefore, the asymptotic confidence interval is

\[\begin{align*} \mathrm{ACI}_{1-\alpha}(\lambda)=\left[\hat{\lambda}_{\mathrm{MLE}}\mp z_{\alpha/2}\frac{\hat{\lambda}_{\mathrm{MLE}}}{\sqrt{n}}\right]. \end{align*}\]

To illustrate this confidence interval, let us generate a simulated sample of size \(n=100\) for \(\lambda=2\) and compute the asymptotic \(95\%\)-confidence interval:

# Sample from Exp(2)
set.seed(123456)
n <- 100
x <- rexp(n = n, rate = 2)

# MLE
lambda_mle <- 1 / mean(x)

# Asymptotic confidence interval
alpha <- 0.05
z <- qnorm(alpha / 2, lower.tail = FALSE)
lambda_mle + c(-1, 1) * z * lambda_mle / sqrt(n)
## [1] 1.374268 2.044294

In this case, \(\lambda=2\) belongs to the confidence interval.