Chapter 18 Homework 4: Hypothesis Testing: Problems and Solutions

Exercise 18.1 (Casella and Berger 8.5) A random sample, $X_1,\cdots,X_n$, is drawn from a Pareto population with p.d.f. $$\begin{equation} f(x|\theta,\nu)=\frac{\theta\nu^{\theta}}{x^{\theta+1}}I_{[\nu,\infty)}(x),\quad \theta>0,\nu>0 \tag{18.1} \end{equation}$$

1. Find the MLEs of $\theta$ and $\nu$.

2. Show that the LRT of $H_0:\theta=1,\nu$ unknown, versus $H_1:\theta\neq 1,\nu$ unknown has critrical region of the form $\{\mathbf{x}:T(\mathbf{x})\leq c_1\,or\,T(\mathbf{x})\geq c_2\}$, where $0<c_1<c_2$ and $T=\log[\frac{\prod_{i=1}^nX_i}{(\min_iX_i)^n}]$.

3. Show that, under $H_0$, $2T$ has a chi squared distribution, and find the number of the degrees of freedom.

Proof. (a) The log-likelihood for each sample point is $$\begin{equation} \ell(\theta,\nu|x_i)=\left\{\begin{aligned} &\log(\theta)+\theta\log(\nu)-(\theta+1)\log(x_i) & \quad x_i\geq\nu\\ &0 &\quad o.w. \end{aligned} \right. \tag{18.2} \end{equation}$$ Therefore the total log-likelihood is $$\begin{equation} \ell(\theta,\nu)=\left\{\begin{aligned} & n\log(\theta)+n\theta\log(\nu)-(\theta+1)(\sum_{i=1}^n\log(x_i)) &\quad \min_ix_i\geq\nu\\ & 0 & \quad o.w. \end{aligned} \right. \tag{18.3} \end{equation}$$ Denote two statistics $T_1=\prod_{i=1}^nX_i$ and $T_2=\min_iX_i$, by noticing that $\theta>0$ and $n>0$, (18.3) is a monotonically increasing function w.r.t. $\nu$, which attains its maximum when $\nu$ attains the upper boundary $T_2$. Thus, the MLE of $\nu$ is $\hat{\nu}=\min_ix_i$. Meanwhile, (18.3) take derivatives w.r.t. $\theta$ we have $$\begin{equation} \frac{\partial\ell}{\partial\theta}=\frac{n}{\theta}+n\log(\nu)-log(T_1) \tag{18.4} \end{equation}$$ (18.4) is a decreasing function w.r.t. $\theta$ so $\ell(\theta,\nu)$ attains the maximum for $\theta$ when (18.4) is setting to 0. Thus, $\hat{\theta}=\frac{n}{log(\prod_{i=1}^nx_i)-n\log(\hat{\nu})}$.

2. Noticing that $\hat{\theta}=\frac{n}{T}$, consider log-likelihood function we have $$\begin{equation} \sup_{\theta\in\Theta}\ell(\boldsymbol(\theta)|\mathbf{x})=nlog(\hat{\theta})+n\hat{\theta}\log(\hat{\nu})-(\hat{\theta}+1)(\log(T_1)) \tag{18.5} \end{equation}$$
   and since $\Theta_0=\{1\}\times\mathbb{R}_+$, we have $$\begin{equation} \sup_{\theta\in\Theta_0}\ell(\boldsymbol(\theta)|\mathbf{x})=n\log(\hat{\nu})-2(\log(T_1)) \tag{18.6} \end{equation}$$ Thus, the test statistic is \begin{eqaution} ()=()^ne{-T+n} \tag{18.7} \end{equation} Taking derivatives w.r.t. $T$ we have $\frac{\partial}{\partial T}\log(\lambda(\mathbf{x}))=(n/T)-1$, hence $\lambda(\mathbf{x})$ is increasing if $T\leq n$ and decreasing if $T\geq n$. Thus, $T\leq c$ is equivalent to $T(\mathbf{x})\leq c_1\,or\,T(\mathbf{x})\geq c_2$ for some $c_1$, $c_2$.

3. Make the following transformation. Let $Y_i=\log X_i$, $i=1,\cdots,n$. Then let $Z_1=\min_iY_i$ and let $Z_2,\cdots,Z_n$ equal to the remaining $Y_i$s. Finally, let $W_1=Z_1$ and $W_i=Z_i-Z_1$, $i=2,\cdots,n$. Then $W_i$s are independent with $W_1\sim f_{W_1}(\omega)=n\nu^n\exp(-n\omega),\omega>\log\nu$ and $\omega_i\sim Exp(1)$, $i=2,\cdots,n$. Now let $T=\sum_{i=2}^nW_i\sim Gamma(n-1,1)$, and hence $2T\sim Gamma(n-1,2)=\chi^2_{2(n-1)}$. The degrees of freedom is $2(n-1)$.

Exercise 18.2 (Casella and Berger 8.6) Suppose that we have two independent random samples: $X_1,\cdots,X_n$ are $Exp(\theta)$ and $Y_1,\cdots,Y_m$ are $Exp(\mu)$.

1. Find the LRT of $H_0:\theta=\mu$ versus $H_1:\theta\neq\mu$.

2. Show that the test in part (a) can be based on the statistic $T=\frac{\sum_{i=1}^nX_i}{\sum_{i=1}^nX_i+\sum_{i=1}^mY_i}$.

3. Find the distribution of $T$ when $H_0$ is true.

Proof. (a) The joint likelihood function is $$\begin{equation} L(\theta,\mu|\mathbf{x},\mathbf{y})=\theta^n\mu^m\exp(-(\theta\sum_{i=1}^nx_i+\mu\sum_{i=1}^my_i)) \tag{18.8} \end{equation}$$ Thus, we have the MLE of (18.8) as $(\hat{\theta},\hat{\mu})=(\frac{n}{\sum_{i=1}^nx_i},\frac{m}{\sum_{i=1}^my_i})$. The maximum value is $(\frac{n}{\sum_{i=1}^nx_i})^n(\frac{m}{\sum_{i=1}^my_i})^m\exp(-(n+m))$. Meanwhile, for $\theta=\mu$, the likelihood function is $\theta^(n+m)\exp(-\theta(\sum_{i=1}^nx_i+\sum_{i=1}^my_i))$ which attains its maximum at $\hat{\theta}_*=\frac{n+m}{\sum_{i=1}^nx_i+\sum_{i=1}^my_i}$. The maximum value is $(\frac{n+m}{\sum_{i=1}^nx_i+\sum_{i=1}^my_i})^{n+m}\exp(-(n+m))$. Hence, $$\begin{equation} \lambda(\mathbf{x},\mathbf{x})=\frac{(n+m)^{n+m}}{n^nm^m}\frac{(\sum_{i=1}^nx_i)^n(\sum_{i=1}^my_i)^m}{(\sum_{i=1}^nx_i+\sum_{i=1}^my_i)^{n+m}} \tag{18.9} \end{equation}$$ The LRT rejects $H_0$ if $\lambda(\mathbf{x},\mathbf{x})\leq c$.

2. Since $$\begin{equation} \frac{(n+m)^{n+m}}{n^nm^m}\frac{(\sum_{i=1}^nx_i)^n(\sum_{i=1}^my_i)^m}{(\sum_{i=1}^nx_i+\sum_{i=1}^my_i)^{n+m}}=\frac{(n+m)^{n+m}}{n^nm^m}T^n(1-T)^m \tag{18.10} \end{equation}$$ (18.10) is a unimode function of $T$ which is maximized when $T=\frac{n}{m+n}$. The rejection region of $\lambda\leq c$ is equivalent to $T\leq a$ or $T\geq b$.

3. Suppose $\theta=\mu=\theta_0$, since $\sum_{i=1}^nX_i\sim Gamma(n,\theta_0)$ and $\sum_{i=1}^mY_i\sim Gamma(m,\theta_0)$. Thus, $T\sim Beta(n,m)$.

Exercise 18.3 (Casella and Berger 8.7) We have already seen the usefulness of the LRT in dealing with problems with nuisance parameters. We now look at some other nuisance parameter problems.

1. Find the LRT of $H_0:\theta\leq 0$ versus $H_1:\theta>0$ based on sample $X_1,\cdots,X_n$ from a population with p.d.f. $f(x|\theta,\lambda)=\frac{1}{\lambda}e^{-(x-\theta)/\lambda}I_{[\theta,\infty)}(x)$, where both $\theta$ and $\lambda$ are unknown.

2. We have previously seen that the exponential p.d.f. is a special case of a Gamma p.d.f. Generalizing in another way, the exponential p.d.f. can be considered as a special case of the $Weibull(\gamma,\beta)$. The Weibull pdf, which reduces to the exponential if $\gamma=1$, is very important in modeling reliability of systems. Suppose that $X_1,\cdots,X_n$ is a random sample from a Weibull population with both $\gamma$ and $\beta$ unknwon. Find the LRT of $H_0:\gamma=1$ versus $H_1:\gamma\neq 1$.

Proof. (a) The joint likelihood is of the form \begin{eqaution} \begin{split} L(,|)&=^{-n}exp(-{i=1}^n(x_i-))I{[,)}(ix_i)\ &=^{-n}exp(-)exp()I{[,)}(_ix_i) \end{split} \tag{18.11} \end{equation} which is globally maximized at $(\hat{\theta},\hat{\lambda})=(\min_ix_i,\bar{x}-\min_ix_i)$. Under the constrain, $\hat{\theta}_*=0$ if $\min_ix_i>0$ and $\hat{\theta}_*=\min_ix_i$ if $\min_ix_i\leq0$. For $\min_ix_i>0$, we have $\hat{\lambda}_*=\bar{x}$. Thus, $$\begin{equation} \lambda(\mathbf{x})=\left\{\begin{aligned} & 1 & \quad \min_ix_i\leq0\\ & (1-\frac{\min_ix_i}{\bar{x}})^n &\quad \min_ix_i>0 \end{aligned} \right. \tag{18.12} \end{equation}$$ So rejecting if $\lambda(\mathbf{x})\leq c$ is equivalent to rejecting if $\frac{\min_ix_i}{\bar{x}}\geq c^*$ for some constant $c^*$.

2. The LRT statistic is $$\begin{equation} \lambda(\mathbf{x})=\frac{\sup_{\beta}(1/\beta^n)exp(-\sum_{i=1}^nx_i/\beta)}{\sup_{\beta,\gamma}(\gamma^n/\beta^n)(\prod_{i=1}^nx_i)^{\gamma-1}exp(-\sum_{i=1}^nx_i^{\gamma}/\beta)} \tag{18.13} \end{equation}$$ The numerator is maximized at $\hat{\beta}_0=\bar{x}$. For fixed $\gamma$, the denominator is maximized at $\hat{\beta}_{\gamma}=\sum_{i=1}^nx_i^{\gamma}/n$. Thus $$\begin{equation} \lambda(\mathbf{x})=\frac{\bar{x}^{-n}}{\sup_{\gamma}(\gamma^n/\hat{\beta}_{\gamma}^n)(\prod_{i=1}^nx_i)^{\gamma-1}} \tag{18.14} \end{equation}$$ The denominator cannot be maximized in close form. Numeric maximization could be used to compute the statistic for observed data $\mathbf{x}$.

Exercise 18.4 (Casella and Berger 8.12) For samples of size $n=1,4,16,64,100$ from a normal population with mean $\mu$ and known variance $\sigma^2$, plot the power function of the following LRTs. Take $\alpha=0.05$.

1. $H_0:\mu\leq 0$ versus $H_1:\mu>0$.

2. $H_0:\mu= 0$ versus $H_1:\mu\neq0$.

Proof. (a) The LRT reject $H_0$ if $\bar{x}>c\sigma/\sqrt{n}$. For $\alpha=0.05$ take $c=1.645$, the power function is $$\begin{equation} \beta(\mu)=P(\frac{\bar{X}-\mu}{\sigma/\sqrt{n}}>1.645-\frac{\mu}{\sigma/\sqrt{n}})=P(Z>1.645-\frac{\sqrt{n}\mu}{\sigma}) \tag{18.15} \end{equation}$$

2. The LRT is to reject $H_0$ if $|\bar{x}|>\frac{c\sigma}{\sqrt{n}}$. For $\alpha=0.05$, $c=1.96$, The power function is
   $$\begin{equation} \beta(\mu)=P(Z<-1.96-\sqrt{n}\mu/\sigma)+P(Z>1.96-\sqrt{n}\mu/\sigma) \tag{18.16} \end{equation}$$

The power function for both cases is shown in Figure 18.1.

FIGURE 18.1: Power function for both tests. The left panel shows the power function of the one side test while the right panel shows the power function for two sides test.

Exercise 18.5 (Casella and Berger 8.14) For a random sample $X_1,\cdots,X_n$ of $Bernoulli(p)$ variables, it is desired to test $H_0:p=0.49$ versus $H_1:p=0.51$. Use the Central Limit Theorem to determine, approximately, the sample size needed so that the two probabilities of error are both about 0.01. Use a test function that rejects $H_0$ if $\sum_{i=1}^nX_i$ is large.

Proof. By CLT, $Z=\frac{\sum_{i=1}^nX_i-np}{\sqrt{np(1-p)}}$ is approximately $N(0,1)$. For a test that rejects $H_0$ when $\sum_{i=1}^nX_i>c$, we need to find $c$ and $n$ to satisfy $$\begin{equation} \begin{split} & P(Z>\frac{c-0.49n}{\sqrt{0.49\cdot 0.51n}})=0.01\\ & P(Z>\frac{c-0.51n}{\sqrt{0.49\cdot 0.51n}})=0.99 \end{split} \tag{18.17} \end{equation}$$ Noticing $z_{0.99}=2.33$, we can solve and get $n=13567$ and $c=6783.5$.

Exercise 18.6 (Casella and Berger 8.15) Show that for a random sample $X_1,\cdots,X_n$ from a $N(0,\sigma^2)$ population, the most powerful test of $H_0:\sigma=\sigma_0$ versus $H_1:\sigma=\sigma_1$, where $\sigma_0<\sigma_1$, is given by $$\begin{equation} \phi(\sum_{i=1}^nX_i^2)=\left\{\begin{aligned} 1 &\quad \sum_{i=1}^nX_i^2>c\\ 0 &\quad \sum_{i=1}^nX_i^2\leq c \end{aligned} \right. \tag{18.18} \end{equation}$$ For a given value of $\alpha$, the size of the Type I Error, show how the value of $c$ is explicitly determined.

Proof. From the Neyman-Pearson lemma, the UMP test rejects $H_0$ if $$\begin{equation} \begin{split} \frac{f(x|\sigma_1)}{f(x|\sigma_0)}&=\frac{(2\pi\sigma_1^2)^{-n/2}exp(-\sum_{i=1}^nx_i^2/(2\sigma_1^2))}{(2\pi\sigma_0^2)^{-n/2}exp(-\sum_{i=1}^nx_i^2/(2\sigma_0^2))}\\ &=(\frac{\sigma_0}{\sigma_1})^n\exp\{\frac{1}{2}\sum_{i=1}^nx_i^2(\frac{1}{\sigma_0^2}-\frac{1}{\sigma_1^2})\}>k \end{split} \tag{18.19} \end{equation}$$ for some $k\geq 0$. This is equivalent to rejecting if $$\begin{equation} \sum_{i=1}^nx_i^2>\frac{2\log(k(\sigma_1/\sigma_0)^n)}{\frac{1}{\sigma_0^2}-\frac{1}{\sigma_1^2}}=c \tag{18.20} \end{equation}$$ This is the UMP test of size $\alpha$, where $\alpha=P_{\sigma_0}(\sum_{i=1}^nX_i^2>c)$. Since $\sum_{i=1}^nX_i^2/\sigma_0^2\sim\chi_n^2$. We have $$\begin{equation} \alpha==P_{\sigma_0}(\sum_{i=1}^nX_i^2>c/\sigma_0^2)=P(\chi_n^2>c/\sigma_0^2) \tag{18.21} \end{equation}$$ so $c=\sigma_0^2\chi^2_{n,\alpha}$

Exercise 18.7 (Casella and Berger 8.18) Let $X_1,\cdots,X_n$ be a random sample from a $N(\theta,\sigma^2)$ population, $\sigma^2$ known. An LRT of $H_0:\theta=\theta_0$ versus $H_1:\theta\neq\theta_0$ is a test that rejects $H_0$ if $\frac{|\bar{X}-\theta_0|}{\sigma/\sqrt{n}}>c$.

1. Find an expression, in terms of standard normal probabilities, for the power function of this test.

2. The experimenter desires a Type I Error probability of 0.05 and a maximum Type II Error probability of 0.25 at $\theta=\theta_0+\sigma$. Find values of $n$ and $c$ such that will achieve this.

Proof. (a) By definition of power function, we have

2. The size is $0.05=\beta(\theta_0)=1+\Phi(-c)-\Phi(c)$ which implies that $c=1.96$. Meanwhile, we have $$\begin{equation} 0.75\leq\beta(\theta_0+\sigma)=1+\Phi(-c-\sqrt{n})-\Phi(c-\sqrt{n})=1+\Phi(-1.96-\sqrt{n})-\Phi(1.96-\sqrt{n}) \tag{18.22} \end{equation}$$ Since $\Phi(-1.96-\sqrt{n})\approx 0$ and $\Phi(-0.675)\approx 0.25$, we have $1.96-\sqrt{n}=-0.675$ and thus $n=6.843\approx 7$.

Exercise 18.8 (Casella and Berger 8.22) Let $X_1,\cdots,X_{10}$ be i.i.d. $Bernoulli(p)$.

1. Find the most powerful test of size $\alpha=0.0547$ of the hypotheses $H_0:p=\frac{1}{2}$ versus $H_1:p=\frac{1}{4}$. Find the power of this test.

2. For testing $H_0:p\leq\frac{1}{2}$ versus $H_1:p>\frac{1}{2}$, find the size and sketch the power function of the test that rejects $H_0$ if $\sum_{i=1}^{10}X_i\geq 6$.

3. For what $\alpha$ levels does there exist a UMP test of the hypotheses in part (a)?

Proof. (a) The test can be based on the sufficient statisitc $\sum_{i=1}^nX_i$. Let $Y=\sum_{i=1}^nX_i\sim Bin(10,p)$ and let $f(y|p)$ denote the p.m.f. of $Y$. By Corollary 16.1, a test that rejeccts if $\frac{f(y|\frac{1}{4})}{f(y|\frac{1}{2})}>k$ is UMP of its size. This ratio is decreasing in y, and rejecting for large value of the ratio is equivalent to rejecting for small values of y. To get $\alpha=0.0547$, we must find c such that $P(Y\leq c|p=\frac{1}{2})=0.0547$ and trying values for $c=0,1,\cdots$ we find that for $c=2$, $P(Y\leq 2|p=\frac{1}{2})=0.0547$. So the test that rejects if $Y\leq 2$ is the UMP $\alpha=0.0547$ test. The power of the test is $P(Y\leq 2|p=\frac{1}{4})\approx 0.526$.

2. The size of the test is $P(Y\geq 6|p=\frac{1}{2})=\sum_{k=6}^{10}{{10} \choose k}(\frac{1}{2})^k(\frac{1}{2})^{10-k}\approx 0.377$. The power function is $\beta(\theta)=\sum_{k=6}^{10}{{10} \choose k}\theta^k(1-\theta)^{10-k}$, which is shown in Figure 18.2.

3. There is a nonrandomized UMP test for all $\alpha$ levels corresponding to the probabilities $P(Y\leq i|p=\frac{1}{2})$, where $i$ is an integer. For $n=10$, $\alpha$ can have any of the values $0,\frac{1}{1024},\frac{11}{1024},\frac{56}{1024},\frac{176}{1024},\frac{386}{1024},\frac{638}{1024},\frac{848}{1024},\frac{968}{1024},\frac{1013}{1024},\frac{1023}{1024}$ and 1.

FIGURE 18.2: Power function for the test.

Exercise 18.9 (Casella and Berger 8.27) Suppose $g(t|\theta)=h(t)c(\theta)e^{\omega(\theta)t}$ is a one-parameter exponential family for the random variable $T$. Show that this family has an MLR if $\omega(\theta)$ is an increasing function of $\theta$. Give three examples of such a family.

Proof. For $\theta_2>\theta_1$ we have $$\begin{equation} \frac{g(t|\theta_2)}{g(t|\theta_1)}=\frac{c(\theta_2)}{c(\theta_1)}exp[(\omega(\theta_2)-\omega(\theta_1))t] \tag{18.23} \end{equation}$$
which is increasing in $t$ becasue $\omega(\theta_2)-\omega(\theta_1)>0$. Example include $N(\theta,1)$, $Beta(\theta,1)$ and $Bernoulli(\theta)$.

Exercise 18.10 (Casella and Berger 8.37) Let $X_1,\cdots,X_n$ be random sample from a $N(\theta,\sigma^2)$. Consider testing $H_0:\theta\leq\theta_0$ versus $H_1:\theta>\theta_0$.

1. If $\sigma^2$ is known, show that the test rejects $H_0$ when $\bar{X}>\theta_0+z_{\alpha}\sqrt{\sigma^2/n}$ is a test of size $\alpha$. Show that the test can be derived as an LRT.

2. Show that the test in part (a) is a UMP test.

3. If $\sigma^2$ is unknown, show that the test that rejects $H_0$ when $\bar{X}>\theta_0+t_{n-1,\alpha}\sqrt{S^2/n}$ is a test of size $\alpha$. Show that the test can be derived as an LRT.

Proof. (a) Since we have $$\begin{equation} P_{\theta_0}(\bar{X}>\theta_0+z_{\alpha}\sqrt{\sigma^2/n})=P_{\theta_0}(\frac{\bar{X}-\theta_0}{\sigma/\sqrt{n}}>z_{\alpha})=P(Z>z_{\alpha})=alpha \tag{18.24} \end{equation}$$ For LRT, since $\bar{x}$ is the unrestricted MLE, and the restricted MLE is $\theta_0$ if $\bar{x}>\theta_0$, we have $$\begin{equation} \lambda(\mathbf{x})=\frac{(2\pi\sigma^2)^{-n/2}\exp[-sum_{i=1}^n(x_i-\theta_0)^2/2\sigma^2]}{(2\pi\sigma^2)^{-n/2}\exp[-sum_{i=1}^n(x_i-\bar{x})^2/2\sigma^2]}=exp(-\frac{n(\bar{x}-\theta_0)^2}{2\sigma^2}) \tag{18.25} \end{equation}$$ and for $\bar{x}<\theta_0$ the LRT statistic is 1. Thus, rejecting if $\lambda<c$ is equivalent to rejecting if $\frac{\bar{x}-\theta_0}{\sigma/\sqrt{n}}>c^{\prime}$.

2. The test is UMP by the Karlin-Rubin Theorem.

3. Since we have $$\begin{equation} P_{\theta_0}(\bar{X}>\theta_0+t_{n-1,\alpha}\sqrt{S^2/n})=P(T_{n-1}>t_{n-1,\alpha})=\alpha \tag{18.26} \end{equation}$$ Denote $\hat{\sigma}^2=\frac{1}{n}\sum_{i=1}^n(x_i-\bar{x})^2$ and $\hat{\sigma}_0^2=\frac{1}{n}(x_i-\theta_0)^2$ then for $\bar{x}\geq\theta_0$ the LRT statistic is $\lambda=(\frac{\hat{\sigma}^2}{\hat{\sigma}^2_0})^{n/2}$ and for $\bar{x}<\theta_0$ the LRT statistic is $\lambda=1$. Since $\hat{\sigma}^2=\frac{n-1}{n}s^2$ and $\hat{\sigma}^2_0=(\bar{x}-\theta_0)^2+\frac{n-1}{n}s^2$, it is clear that the LRT is equivalent to the $t-$test becasue $\lambda<c$ when $$\begin{equation} \frac{\frac{n-1}{n}s^2}{(\bar{x}-\theta_0)^2+\frac{n-1}{n}s^2}=\frac{(n-1)/n}{(\bar{x}-\theta_0)^2/s^2+(n-1)/n}<c^{\prime} \tag{18.27} \end{equation}$$ and $\bar{x}\geq\theta_0$, which is the same as rejecting when $\frac{\bar{x}-\theta_0}{s/\sqrt{n}}$ is large.

Exercise 18.11 (Casella and Berger 8.38) Let $X_1,\cdots,X_n$ be i.i.d. $N(\theta,\sigma^2)$, where $\theta_0$ is a specified value of $\theta$ and $\sigma^2$ is unknown. We are interested in testing $H_0:\theta=\theta_0$ versus $H_1:\theta\neq\theta_0$.

1. Show that the test rejects $H_0$ when $|\bar{X}-\theta_0|>t_{n-1,\alpha/2}\sqrt{S^2/n}$ is a test of size $\alpha$.

2. Show that the test in part (a) can be derived as an LRT.

Proof. (a) For the size we have $$\begin{equation} \begin{split} &P_{\theta_0}\{|\bar{X}-\theta_0|>t_{n-1,\alpha/2}\sqrt{S^2/n}\}\\ &=1-P_{\theta_0}\{-t_{n-1,\alpha/2}\sqrt{S^2/n}\leq \bar{X}-\theta_0\leq t_{n-1,\alpha/2}\sqrt{S^2/n}\}\\ &=1-P_{\theta_0}\{-t_{n-1,\alpha/2}\leq\frac{\bar{X}-\theta_0}{\sqrt{S^2/n}}\leq t_{n-1,\alpha/2}\}\\ &=1-(1-\alpha)=\alpha \end{split} \tag{18.28} \end{equation}$$ as we desired.

2. The unrestriced MLEs are $\hat{\theta}=\bar{X}$ and $\hat{\sigma}^2=\sum_{i=1}^n(X_i-\bar{X})^2/n$. The restricted MLEs are $\hat{\theta}_0=\theta_0$ and $\hat{\sigma}_0^2=\sum_{i=1}^n(X_i-\theta_0)^2/n$. So the LRT statistic is $$\begin{equation} \lambda(\mathbf{x})=[\frac{\sum_{i=1}^n(x_i-\bar{x})^2}{\sum_{i=1}^n(x_i-\bar{x})^2+n(\bar{x}-\theta_0)^2}]^{n/2} \tag{18.29} \end{equation}$$ Using the same algebra as part (c) of Exercise 18.10 we can derive the LRT as rejection when $$\begin{equation} |\bar{x}-\theta_0|>[(c^{-2/n}-1)(n-1)\frac{s^2}{n}]^{1/2} \tag{18.30} \end{equation}$$ choose constant $c$ to achieve size $\alpha$, we will have the same $t$ test.

Exercise 18.12 (Casella and Berger 8.41) Let $X_1,\cdots,X_n$ be a random sample from a $N(\mu_{X},\sigma_X^2)$, and let $Y_1,\cdots,Y_m$ be an independent random sample from a $N(\mu_{Y},\sigma_Y^2)$. We are interested in testing $H_0:\mu_X=\mu_Y$ versus $H_1:\mu_X\neq\mu_Y$ with assumption that $\sigma^2_X=\sigma^2_Y=\sigma^2$.

1. Derive the LRT for these hypotheses. Show that the LRT can be based on the statistic $T=\frac{\bar{X}-\bar{Y}}{\sqrt{S^2_p(\frac{1}{n}+\frac{1}{m})}}$, where $S_p^2=\frac{1}{n+m-2}(\sum_{i=1}^n(X_i-\bar{X})^2+\sum_{i=1}^m(Y_i-\bar{Y})^2)$.

2. Show that, under $H_0$, $T\sim t_{n+m-2}$. (This test is known as the two-sample t test.)

3. Given the following two groups of data, use the two-sample t test to determine if the two group have the same mean.

Group 1: 1294, 1279, 1274, 1264, 1263, 1254, 1251, 1251, 1248, 1240, 1232, 1220, 1218, 1210

Group 2: 1284, 1272, 1256, 1254, 1242, 1274, 1264, 1256, 1250

Proof. (a) By LRT $$\begin{equation} \lambda(\mathbf{x},\lambda{y})=\frac{L(\hat{mu},\hat{\sigma}_0^2|\mathbf{x},\mathbf{y})}{L(\hat{\mu}_X,\hat{\mu}_Y,\hat{\sigma}_1^2|\mathbf{x},\mathbf{y})} \tag{18.31} \end{equation}$$ Under $H_0$, $\hat{\mu}=\frac{n\bar{x}+n\bar{y}}{n+m}$ and $\sigma^2_0=\frac{\sum_{i=1}^n(X_i-\hat{\mu})^2+\sum_{i=1}^n(Y_i-\hat{\mu})^2}{n+m}$. To obtain the unrestricted MLEs, taking derivatives for $L(\hat{\mu}_X,\hat{\mu}_Y,\hat{\sigma}_1^2|\mathbf{x},\mathbf{y})$ we will have $\hat{\mu}_X=\bar{x}$, $\hat{\mu}=\bar{y}$ and $\hat{\sigma}_1^2=\frac{1}{n+m}(\sum_{i=1}^n(X_i-\bar{X})^2+\sum_{i=1}^m(Y_i-\bar{Y})^2)$. Thus, by simple algebra we have $$\begin{equation} \lambda(\mathbf{x},\lambda{y})=(\frac{\hat{\sigma}_0^2}{\hat{\sigma}_1^2})^{-\frac{n+m}{2}} \tag{18.32} \end{equation}$$ Reject for larger values of LRT is then equivalent to reject for large value of $|T|$, by simple algebra.

2. Since $$\begin{equation} T=\frac{\bar{X}-\bar{Y}}{\sqrt{S^2_p(\frac{1}{n}+\frac{1}{m})}}=\frac{(\bar{X}-\bar{Y})/\sqrt{\sigma^2(1/n+1/m)}}{\sqrt{[(n+m-2)S_p^2/\sigma^2]/(n+m-2)}} \tag{18.33} \end{equation}$$ Under $H_0$, since $\bar{X}-\bar{Y} \sim N(0,\sigma^2(1/n+1/m))$ and $(n-1)S_X^2/\sigma^2$, $(m-1)S_Y^2/\sigma^2$ are indpendent $\chi^2$ distributed with degree of freedom $(n-1)$ and $(m-1)$. Hence $(n+m-2)S_p^2/\sigma^2\sim\chi^2_{n+m-2}$. Furthermore, $\bar{X}-\bar{Y}$ and $S_X^2$ and $S_Y^2$ are independent. Thus, $T\sim t_{n+m-2}$.

3. For the data, we have $n=14,\bar{X}=1249.86,S_X^2=591.36,m=9,\bar{Y}=1261.33,S_Y^2=176.00$ and $S_p^2=433.13$. Thus, $T=-1.29$ and comparing to a $t$ distribution with 21 degrees of freedom. We have the p-value is approximately 0.21. So we fail to reject the null hypothesis.

Casella, George, and Roger Berger. 2002. Statistical Inference. 2nd ed. Belmont, CA: Duxbury Resource Center.

Rencher, Alvin, and Bruce Schaalje. 2007. Linear Models in Statistics. 2nd ed. John Wiley; Sons, Ltd.