## Exercises

Exercise 3.1 We have a srs of size $$n$$ from a population with mean $$\mu$$ and variance $$\sigma^2.$$

1. Prove that $$\sum_{i=1}^na_iX_i$$ is an unbiased estimator of $$\mu$$ if $$\sum_{i=1}^na_i=1.$$
2. Among all the unbiased estimators of this form, find the one with minimum variance and compute it.

Exercise 3.2 Let $$(X_1,\ldots,X_n)$$ be a srs from a $$\mathcal{U}\left(0, \theta \right),$$ $$\theta >0.$$ Find out whether the following estimators are unbiased for the population mean and, in case of a positive answer, find their biases:

\begin{align*} \bar{X},\quad X_1,\quad X_{(n)},\quad X_{(1)},\quad 0.5 X_{(n)}+0.5 X_{(1)}. \end{align*}

Exercise 3.3 The consumption of a certain good in a family with four members during the summer months is a rv with $$\mathcal{U}(\alpha,\alpha+1)$$ distribution. Let $$(X_1,\ldots,X_n)$$ be a srs of consumption of the same good for different families.

1. Show that the sample mean is biased for $$\alpha$$ and that its bias is $$1/2.$$
2. Compute the MSE of $$\bar{X}$$ as an estimator of $$\alpha.$$
3. Obtain from $$\bar{X}$$ an unbiased estimator of $$\alpha$$ and provide its MSE.

Exercise 3.4 Let $$(X_1,\ldots,X_n)$$ be a srs from an $$\mathrm{Exp}(\lambda).$$

1. Find an unbiased estimator of $$\lambda$$ based on $$X_{(1)}.$$
2. Compare the previous estimator with $$\bar{X}$$ and decide which one is better.
3. The lifetime of a light bulb in days is usually modeled by an exponential distribution. The following data are lifetimes for light bulbs: $$50.1,$$ $$70.1,$$ $$137,$$ $$166.9,$$ $$170.5,$$ $$152.8,$$ $$80.5,$$ $$123.5,$$ $$112.6,$$ $$148.5,$$ $$160,$$ $$125.4.$$ Estimate the average duration of a light bulb using the two previous estimators.

Exercise 3.5 Assume that the rv’s $$Y_1,\ldots,Y_n$$ are such that

\begin{align*} Y_i=\beta x_i+\varepsilon_i,\quad i=1,\ldots,n, \end{align*}

where the $$x_i$$’s are constants and the $$\varepsilon_i$$ are iid rv’s distributed as $$\mathcal{N}(0,\sigma^2).$$

1. Prove that $$\sum_{i=1}^nY_i/\sum_{i=1}^nx_i$$ is an unbiased estimator of $$\beta.$$
2. Prove that $$(1/n)\sum_{i=1}^nY_i/x_i$$ is also unbiased.
3. Compute the variances of both estimators.

Exercise 3.6 Let $$(X_1,\ldots,X_{n_1})$$ be a srs of $$X$$ with $$\mathbb{E}[X]=\mu_1$$ and $$\mathbb{V}\mathrm{ar}[X]=\sigma^2,$$ and let $$(Y_1,\ldots ,Y_{n_2})$$ be a srs of $$Y$$ with $$\mathbb{E}[Y]=\mu_2$$ and $$\mathbb{V}\mathrm{ar}[Y]=\sigma^2,$$ with $$X$$ and $$Y$$ independent.

1. Prove that $$S^2:=wS_1'^2+(1-w)S_2'^2$$ is a consistent estimator of $$\sigma^2,$$ for any $$w\in(0,1).$$
2. Show that

\begin{align*} \frac{\bar{X}-\bar{Y}-(\mu_1-\mu_2)}{S\sqrt{{\frac{1}{n_1}}+{\frac{1}{n_2}}}}\stackrel{d}{\longrightarrow} \mathcal{N}(0,1). \end{align*}

Exercise 3.7 The number of independent arrivals to an emergency service during a day follows a $$\mathrm{Pois}(\lambda)$$ distribution. In order to estimate $$\lambda$$ and forecast the amount of required personnel in the service, we observe the arrivals during $$n$$ days. We know that $$T=\sum_{i=1}^nX_i$$ is a sufficient statistic. Determine:

1. $$k>0$$ such that the estimator $$T_k=kT$$ is unbiased for $$\lambda.$$
2. The condition that the sequence $$k_n$$ must satisfy for obtaining a sequence of estimators that is consistent in probability, if the sample size $$n$$ is allowed to grow.

Exercise 3.8 Consider a sample as in Exercise 3.2.

1. Prove that $$X_{(n)}$$ is consistent in probability for $$\theta.$$
2. Check whether $$Y_n=2\bar{X}_n$$ is also consistent in probability for $$\theta.$$

Exercise 3.9 Let $$(X_1,\ldots,X_{10})$$ be a srs from a distribution with mean $$\mu$$ and variance $$\sigma^2.$$ Consider the following two estimators of $$\mu$$:

\begin{align*} \hat{\mu}_1=X_1+\frac{1}{2}X_2-\frac{1}{2}X_3, \quad \hat{\mu}_2=X_4+\frac{1}{5}X_5-\frac{1}{10}X_{10}. \end{align*}

Which one is better in terms of MSE? Are they consistent in probability?

Exercise 3.10 Let $$(X_1,\ldots,X_n)$$ be a srs from a distribution with mean $$\mu$$ and variance $$\sigma^2.$$ Consider the following estimators of $$\mu$$:

\begin{align*} \hat{\mu}_1=\frac{X_1+2X_2+3X_3}{6}, \quad \hat{\mu}_2=\frac{X_1+4X_2+X_3}{6}, \quad \hat{\mu}_3=\frac{\frac{3}{2}X_1+\frac{1}{2}X_2+X_3+\cdots+X_n}{n}. \end{align*}

1. Which ones are unbiased?
2. Among the unbiased ones, which is the most efficient?
3. Find an unbiased estimator of $$\mu$$ different from $$\bar{X}$$ that is more efficient than the previous unbiased estimators.
4. Which of them is consistent in squared mean?

Exercise 3.11 Let $$(X_1,\ldots,X_n)$$ be a srs of a rv with $$\mathrm{Pois}(\lambda)$$ distribution. Check that

\begin{align*} \frac{\bar{X}-\lambda}{\sqrt{\bar{X}/n}}\stackrel{d}{\longrightarrow}\mathcal{N}(0,1). \end{align*}

Exercise 3.12 Prove the law of the large numbers from Theorem 3.2 using Theorem 3.1.

Exercise 3.13 Let $$(X_1,\ldots,X_n)$$ be a srs from a geometric distribution, whose pmf is

\begin{align*} p(x;\theta) =\theta (1-\theta)^{x-1},\quad x=1,2,\ldots,\ \theta \in (0, 1). \end{align*}

Prove that $$\sum_{i=1}^nX_i$$ is sufficient for $$\theta.$$

Exercise 3.14 Let $$(X_1,\ldots,X_n)$$ be a srs from a population with pdf

\begin{align*} f(x;\theta)=\frac{2x}{\theta}e^{-x^2/\theta} 1_{\{x>0\}},\quad \theta>0. \end{align*}

Show that $$\sum_{i=1}^n X_i^2$$ is sufficient for $$\theta$$ and that $$\sum_{i=1}^n X_i$$ is not.

Exercise 3.15 Let $$(X_1,\ldots,X_n)$$ be a srs from a continuous Bernoulli distribution, whose pdf is

\begin{align*} f(x;\lambda) = c(\lambda) \lambda^x(1-\lambda)^{1-x},\quad x\in[0,1],\ \lambda \in (0,1), \end{align*}

with $$c(\lambda)=21_{\{\lambda=1/2\}}+ \frac{2 \tanh^{-1}(1-2 \lambda)}{1-2 \lambda}1_{\{\lambda\neq 1/2\}}.$$ Find a sufficient statistic for $$\lambda.$$

Exercise 3.16 Let $$X_1$$ be a normal rv with mean zero and variance $$\sigma^2.$$ Is $$|X_1|$$ a sufficient statistic for $$\sigma^2$$?

Exercise 3.17 Let $$(X_1,\ldots,X_n)$$ be a srs from a particular Weibull distribution with pdf

\begin{align*} f(x;\theta)=\frac{2x}{\theta}e^{-x^2/\theta}, \quad x>0,\ \theta>0. \end{align*}

Find a minimal sufficient statistic for $$\theta.$$

Exercise 3.18 Let $$(X_1,\ldots,X_n)$$ be a srs from a Laplace distribution $$\mathrm{Laplace}(\mu,\sigma),$$ whose pdf is

\begin{align*} f(x;\mu,\sigma)=\frac{1}{2\sigma}e^{-|x-\mu|/\sigma}, \quad x\in\mathbb{R},\ \mu\in\mathbb{R},\ \sigma>0. \end{align*}

Find a minimal sufficient statistic for $$\sigma$$ assuming that $$\mu$$ is known.

Exercise 3.19 Let $$(X_1,\ldots,X_n)$$ be a srs from a beta distribution $$\mathrm{Beta}(\alpha,\beta),$$ whose pdf is

\begin{align*} f(x;\alpha,\beta)=\frac{\Gamma(\alpha+\beta)}{\Gamma(\alpha) \Gamma(\beta)} x^{\alpha-1}(1-x)^{\beta-1}, \quad x\in(0,1),\ \alpha>0,\ \beta>0. \end{align*}

Find a minimal sufficient statistic for:

1. $$\alpha,$$ assuming that $$\beta$$ is known.
2. $$\beta,$$ assuming that $$\alpha$$ is known.

Exercise 3.20 Compute the Fisher information for $$X\sim\mathrm{Exp}(\lambda).$$ Then, derive an efficient estimator for $$\lambda.$$

Exercise 3.21 Is it possible to compute the Fisher information for the following distributions?

1. $$\mathcal{U}(0,\theta).$$
2. The pdf $$f(x;\theta)=\theta/x^2,$$ $$x\geq\theta>0.$$

What do these two distributions have in common?

Exercise 3.22 Let $$(X_1,\ldots,X_n)$$ be a srs of a rv $$X\sim\mathcal{N}(\mu,\sigma^2).$$ Show that $$\bar{X}$$ is an efficient estimator of $$\mu.$$

Exercise 3.23 Let $$(X_1,\ldots,X_{n_1})$$ be a srs of $$X\sim \mathcal{N}(\mu_1,\sigma^2)$$ and let $$(Y_1,\ldots,Y_{n_2})$$ be a srs of $$Y\sim \mathcal{N}(\mu_2,\sigma^2),$$ with $$X$$ and $$Y$$ independent. As an estimator of $$\sigma^2,$$ we consider a linear combination of the sample quasivariances $$S_1'^2$$ and $$S_2'^2,$$ that is, $$w S_1'^2 + (1-w)S_2'^2,$$ for $$0\leq w \leq 1.$$

1. Prove that this estimator is unbiased for any value of $$w.$$
2. Obtain the value of $$w$$ that provides the most efficient estimator.

Exercise 3.24 Consider a srs $$(X_1,\ldots,X_n)$$ from a $$\mathrm{Cauchy}(\theta,\sigma)$$ distribution. Use a simulation like that in Figure 3.1 to check empirically if $$\hat{\theta}_n$$ is a consistent estimator for $$\theta.$$ Consider these estimators:

1. $$\hat{\theta}_n=\bar{X}.$$
2. $$\hat{\theta}_n=\mathrm{med}\{X_1,\ldots,X_n\}.$$
3. $$\hat{\theta}_n=T_{0.10}$$ ($$0.10$$-trimmed mean).

Explore in each case $$N=5$$ simulations for each of the three choices of $$(\theta,\sigma)$$ and a sequence of $$n$$’s that increase at least until $$n=10^4.$$ What are your conclusions?

Exercise 3.25 The Fisher information (3.7) is an expectation. Therefore, from a srs $$(X_1,\ldots,X_n)$$ from a continuous rv with pdf $$f(\cdot;\theta),$$ it can be approximated as the sample mean

\begin{align*} \hat{\mathcal{I}}(\theta):=\frac{1}{n}\sum_{i=1}^n \left(\frac{\partial \log f(X_i;\theta)}{\partial \theta}\right)^2. \end{align*}

The construction is analogous for a discrete rv.

Consider a $$\mathrm{Pois}(\lambda)$$ distribution. Use a simulation to check empirically how $$\hat{\mathcal{I}}(\lambda)$$ approximates $$\mathcal{I}(\lambda)=1/\lambda$$ by:

1. Plotting the curve $$\lambda\mapsto\mathcal{I}(\lambda)$$ over the grid lambda <- seq(1, 10, by = 0.5).
2. For each lambda, simulating $$(X_1,\ldots,X_n)\sim\mathrm{Pois}(\lambda)$$ and plotting the curve $$\lambda\mapsto\hat{\mathcal{I}}(\lambda).$$

Use $$n=10,100,1000.$$ What are your conclusions? What if in Step 2 the simulation was done from $$(X_1,\ldots,X_n)\sim\mathrm{Pois}(\lambda_0),$$ $$\lambda_0=2,$$ for each lambda?