Exercises
Exercise 3.1 We have a srs of size \(n\) from a population with mean \(\mu\) and variance \(\sigma^2.\)
- Prove that \(\sum_{i=1}^na_iX_i\) is an unbiased estimator of \(\mu\) if \(\sum_{i=1}^na_i=1.\)
- Among all the unbiased estimators of this form, find the one with minimum variance and compute it.
Exercise 3.2 Let \((X_1,\ldots,X_n)\) be a srs from a \(\mathcal{U}\left(0, \theta \right),\) \(\theta >0.\) Find out whether the following estimators are unbiased for the population mean and, in case of a positive answer, find their biases:
\[\begin{align*} \bar{X},\quad X_1,\quad X_{(n)},\quad X_{(1)},\quad 0.5 X_{(n)}+0.5 X_{(1)}. \end{align*}\]
Exercise 3.3 The consumption of a certain good in a family with four members during the summer months is a rv with \(\mathcal{U}(\alpha,\alpha+1)\) distribution. Let \((X_1,\ldots,X_n)\) be a srs of consumption of the same good for different families.
- Show that the sample mean is biased for \(\alpha\) and that its bias is \(1/2.\)
- Compute the MSE of \(\bar{X}\) as an estimator of \(\alpha.\)
- Obtain from \(\bar{X}\) an unbiased estimator of \(\alpha\) and provide its MSE.
Exercise 3.4 Let \((X_1,\ldots,X_n)\) be a srs from an \(\mathrm{Exp}(\lambda).\)
- Find an unbiased estimator of \(\lambda\) based on \(X_{(1)}.\)
- Compare the previous estimator with \(\bar{X}\) and decide which one is better.
- The lifetime of a light bulb in days is usually modeled by an exponential distribution. The following data are lifetimes for light bulbs: \(50.1,\) \(70.1,\) \(137,\) \(166.9,\) \(170.5,\) \(152.8,\) \(80.5,\) \(123.5,\) \(112.6,\) \(148.5,\) \(160,\) \(125.4.\) Estimate the average duration of a light bulb using the two previous estimators.
Exercise 3.5 Assume that the rv’s \(Y_1,\ldots,Y_n\) are such that
\[\begin{align*} Y_i=\beta x_i+\varepsilon_i,\quad i=1,\ldots,n, \end{align*}\]
where the \(x_i\)’s are constants and the \(\varepsilon_i\) are iid rv’s distributed as \(\mathcal{N}(0,\sigma^2).\)
- Prove that \(\sum_{i=1}^nY_i/\sum_{i=1}^nx_i\) is an unbiased estimator of \(\beta.\)
- Prove that \((1/n)\sum_{i=1}^nY_i/x_i\) is also unbiased.
- Compute the variances of both estimators.
Exercise 3.6 Let \((X_1,\ldots,X_{n_1})\) be a srs of \(X\) with \(\mathbb{E}[X]=\mu_1\) and \(\mathbb{V}\mathrm{ar}[X]=\sigma^2,\) and let \((Y_1,\ldots ,Y_{n_2})\) be a srs of \(Y\) with \(\mathbb{E}[Y]=\mu_2\) and \(\mathbb{V}\mathrm{ar}[Y]=\sigma^2,\) with \(X\) and \(Y\) independent.
- Prove that \(S^2:=wS_1'^2+(1-w)S_2'^2\) is a consistent estimator of \(\sigma^2,\) for any \(w\in(0,1).\)
- Show that
\[\begin{align*} \frac{\bar{X}-\bar{Y}-(\mu_1-\mu_2)}{S\sqrt{{\frac{1}{n_1}}+{\frac{1}{n_2}}}}\stackrel{d}{\longrightarrow} \mathcal{N}(0,1). \end{align*}\]
Exercise 3.7 The number of independent arrivals to an emergency service during a day follows a \(\mathrm{Pois}(\lambda)\) distribution. In order to estimate \(\lambda\) and forecast the amount of required personnel in the service, we observe the arrivals during \(n\) days. We know that \(T=\sum_{i=1}^nX_i\) is a sufficient statistic. Determine:
- \(k>0\) such that the estimator \(T_k=kT\) is unbiased for \(\lambda.\)
- The condition that the sequence \(k_n\) must satisfy for obtaining a sequence of estimators that is consistent in probability, if the sample size \(n\) is allowed to grow.
Exercise 3.8 Consider a sample as in Exercise 3.2.
- Prove that \(X_{(n)}\) is consistent in probability for \(\theta.\)
- Check whether \(Y_n=2\bar{X}_n\) is also consistent in probability for \(\theta.\)
Exercise 3.9 Let \((X_1,\ldots,X_{10})\) be a srs from a distribution with mean \(\mu\) and variance \(\sigma^2.\) Consider the following two estimators of \(\mu\):
\[\begin{align*} \hat{\mu}_1=X_1+\frac{1}{2}X_2-\frac{1}{2}X_3, \quad \hat{\mu}_2=X_4+\frac{1}{5}X_5-\frac{1}{10}X_{10}. \end{align*}\]
Which one is better in terms of MSE? Are they consistent in probability?
Exercise 3.10 Let \((X_1,\ldots,X_n)\) be a srs from a distribution with mean \(\mu\) and variance \(\sigma^2.\) Consider the following estimators of \(\mu\):
\[\begin{align*} \hat{\mu}_1=\frac{X_1+2X_2+3X_3}{6}, \quad \hat{\mu}_2=\frac{X_1+4X_2+X_3}{6}, \quad \hat{\mu}_3=\frac{\frac{3}{2}X_1+\frac{1}{2}X_2+X_3+\cdots+X_n}{n}. \end{align*}\]
- Which ones are unbiased?
- Among the unbiased ones, which is the most efficient?
- Find an unbiased estimator of \(\mu\) different from \(\bar{X}\) that is more efficient than the previous unbiased estimators.
- Which of them is consistent in squared mean?
Exercise 3.11 Let \((X_1,\ldots,X_n)\) be a srs of a rv with \(\mathrm{Pois}(\lambda)\) distribution. Check that
\[\begin{align*} \frac{\bar{X}-\lambda}{\sqrt{\bar{X}/n}}\stackrel{d}{\longrightarrow}\mathcal{N}(0,1). \end{align*}\]
Exercise 3.13 Let \((X_1,\ldots,X_n)\) be a srs from a geometric distribution, whose pmf is
\[\begin{align*} p(x;\theta) =\theta (1-\theta)^{x-1},\quad x=1,2,\ldots,\ \theta \in (0, 1). \end{align*}\]
Prove that \(\sum_{i=1}^nX_i\) is sufficient for \(\theta.\)
Exercise 3.14 Let \((X_1,\ldots,X_n)\) be a srs from a population with pdf
\[\begin{align*} f(x;\theta)=\frac{2x}{\theta}e^{-x^2/\theta} 1_{\{x>0\}},\quad \theta>0. \end{align*}\]
Show that \(\sum_{i=1}^n X_i^2\) is sufficient for \(\theta\) and that \(\sum_{i=1}^n X_i\) is not.
Exercise 3.15 Let \((X_1,\ldots,X_n)\) be a srs from a continuous Bernoulli distribution, whose pdf is
\[\begin{align*} f(x;\lambda) = c(\lambda) \lambda^x(1-\lambda)^{1-x},\quad x\in[0,1],\ \lambda \in (0,1), \end{align*}\]
with \(c(\lambda)=21_{\{\lambda=1/2\}}+ \frac{2 \tanh^{-1}(1-2 \lambda)}{1-2 \lambda}1_{\{\lambda\neq 1/2\}}.\) Find a sufficient statistic for \(\lambda.\)
Exercise 3.16 Let \(X_1\) be a normal rv with mean zero and variance \(\sigma^2.\) Is \(|X_1|\) a sufficient statistic for \(\sigma^2\)?
Exercise 3.17 Let \((X_1,\ldots,X_n)\) be a srs from a particular Weibull distribution with pdf
\[\begin{align*} f(x;\theta)=\frac{2x}{\theta}e^{-x^2/\theta}, \quad x>0,\ \theta>0. \end{align*}\]
Find a minimal sufficient statistic for \(\theta.\)
Exercise 3.18 Let \((X_1,\ldots,X_n)\) be a srs from a Laplace distribution \(\mathrm{Laplace}(\mu,\sigma),\) whose pdf is
\[\begin{align*} f(x;\mu,\sigma)=\frac{1}{2\sigma}e^{-|x-\mu|/\sigma}, \quad x\in\mathbb{R},\ \mu\in\mathbb{R},\ \sigma>0. \end{align*}\]
Find a minimal sufficient statistic for \(\sigma\) assuming that \(\mu\) is known.
Exercise 3.19 Let \((X_1,\ldots,X_n)\) be a srs from a beta distribution \(\mathrm{Beta}(\alpha,\beta),\) whose pdf is
\[\begin{align*} f(x;\alpha,\beta)=\frac{\Gamma(\alpha+\beta)}{\Gamma(\alpha) \Gamma(\beta)} x^{\alpha-1}(1-x)^{\beta-1}, \quad x\in(0,1),\ \alpha>0,\ \beta>0. \end{align*}\]
Find a minimal sufficient statistic for:
- \(\alpha,\) assuming that \(\beta\) is known.
- \(\beta,\) assuming that \(\alpha\) is known.
Exercise 3.20 Compute the Fisher information for \(X\sim\mathrm{Exp}(\lambda).\) Then, derive an efficient estimator for \(\lambda.\)
Exercise 3.21 Is it possible to compute the Fisher information for the following distributions?
- \(\mathcal{U}(0,\theta).\)
- The pdf \(f(x;\theta)=\theta/x^2,\) \(x\geq\theta>0.\)
What do these two distributions have in common?
Exercise 3.22 Let \((X_1,\ldots,X_n)\) be a srs of a rv \(X\sim\mathcal{N}(\mu,\sigma^2).\) Show that \(\bar{X}\) is an efficient estimator of \(\mu.\)
Exercise 3.23 Let \((X_1,\ldots,X_{n_1})\) be a srs of \(X\sim \mathcal{N}(\mu_1,\sigma^2)\) and let \((Y_1,\ldots,Y_{n_2})\) be a srs of \(Y\sim \mathcal{N}(\mu_2,\sigma^2),\) with \(X\) and \(Y\) independent. As an estimator of \(\sigma^2,\) we consider a linear combination of the sample quasivariances \(S_1'^2\) and \(S_2'^2,\) that is, \(w S_1'^2 + (1-w)S_2'^2,\) for \(0\leq w \leq 1.\)
- Prove that this estimator is unbiased for any value of \(w.\)
- Obtain the value of \(w\) that provides the most efficient estimator.
Exercise 3.24 Consider a srs \((X_1,\ldots,X_n)\) from a \(\mathrm{Cauchy}(\theta,\sigma)\) distribution. Use a simulation like that in Figure 3.1 to check empirically if \(\hat{\theta}_n\) is a consistent estimator for \(\theta.\) Consider these estimators:
- \(\hat{\theta}_n=\bar{X}.\)
- \(\hat{\theta}_n=\mathrm{med}\{X_1,\ldots,X_n\}.\)
- \(\hat{\theta}_n=T_{0.10}\) (\(0.10\)-trimmed mean).
Explore in each case \(N=5\) simulations for each of the three choices of \((\theta,\sigma)\) and a sequence of \(n\)’s that increase at least until \(n=10^4.\) What are your conclusions?
Exercise 3.25 The Fisher information (3.7) is an expectation. Therefore, from a srs \((X_1,\ldots,X_n)\) from a continuous rv with pdf \(f(\cdot;\theta),\) it can be approximated as the sample mean
\[\begin{align*} \hat{\mathcal{I}}(\theta):=\frac{1}{n}\sum_{i=1}^n \left(\frac{\partial \log f(X_i;\theta)}{\partial \theta}\right)^2. \end{align*}\]
The construction is analogous for a discrete rv.
Consider a \(\mathrm{Pois}(\lambda)\) distribution. Use a simulation to check empirically how \(\hat{\mathcal{I}}(\lambda)\) approximates \(\mathcal{I}(\lambda)=1/\lambda\) by:
- Plotting the curve \(\lambda\mapsto\mathcal{I}(\lambda)\) over the grid
lambda <- seq(1, 10, by = 0.5). - For each
lambda, simulating \((X_1,\ldots,X_n)\sim\mathrm{Pois}(\lambda)\) and plotting the curve \(\lambda\mapsto\hat{\mathcal{I}}(\lambda).\)
Use \(n=10,100,1000.\) What are your conclusions? What if in Step 2 the simulation was done from \((X_1,\ldots,X_n)\sim\mathrm{Pois}(\lambda_0),\) \(\lambda_0=2,\) for each lambda?