3.2 Invariant estimators

Let us introduce the concept of invariance with an example.

Example 3.7 The manufacturer of a given product claims that the product packages contain at least $$\theta$$ grams of product. If this claim is true, then the product content within a package is distributed as $$\mathcal{U}(\theta, \theta+100).$$ To check the claim of the manufacturer, a srs measuring the content was taken. The realization of this srs is $$(x_1,\ldots,x_n)$$ and is used to compute an estimate $$\hat{\theta}(x_1,\ldots,x_n).$$ But, after computing the estimate, it is discovered that the balance was weighing systematically $$c$$ grams less. Can we just simply correct the estimate as $$\hat{\theta}(x_1,\ldots,x_n)+c$$?

The answer to the question depends on whether the estimator verifies

\begin{align*} \hat{\theta}(x_1+c,\ldots,x_n+c)=\hat{\theta}(x_1,\ldots,x_n)+c. \end{align*}

If this is not the case, then we will need to compute $$\hat{\theta}(x_1+c,\ldots,x_n+c)$$ without being able to reuse $$\hat{\theta}(x_1,\ldots,x_n).$$

Definition 3.3 (Translation-invariant estimator) An estimator $$\hat{\theta}$$ is translation-invariant if, for sample realization $$(x_1,\ldots,x_n)$$ and any $$c\in\mathbb{R},$$

\begin{align*} \hat{\theta}(x_1+c,\ldots,x_n+c)=\hat{\theta}(x_1,\ldots,x_n)+c. \end{align*}

Example 3.8 Check that $$X_{(1)},$$ $$\bar{X},$$ and $$(X_{(1)}+X_{(n)})/2$$ are statistics invariant to translations, but that the geometric mean $$(\prod_{i=1}^n X_i)^{1/n}$$ and the harmonic mean $$n/\sum_{i=1}^n X_i^{-1}$$ are not.

For $$X_{(1)}=\min_{1\leq i\leq n} X_i,$$ we have

\begin{align*} \min_{1\leq i\leq n} (X_i+c)=\min_{1\leq i\leq n} (X_i)+c. \end{align*}

Therefore, $$X_{(1)}$$ is translation-invariant. For $$\bar{X}=(1/n)\sum_{i=1}^n X_i,$$

\begin{align*} \frac{1}{n}\sum_{i=1}^n (X_i+c)=\frac{1}{n}\left[\sum_{i=1}^n X_i+nc\right]=\frac{1}{n}\sum_{i=1}^n X_i+c=\bar{X}+c, \end{align*}

so $$\bar{X}$$ is translation-invariant too. We now check $$(X_{(1)}+X_{(n)})/2$$:

\begin{align*} \frac{1}{2}\left[\min_{1\leq i\leq n}(X_i+c)+\max_{1\leq i\leq n}(X_i+c)\right]&=\frac{1}{2}\left[\min_{1\leq i\leq n}X_i+\max_{1\leq i\leq n}X_i+2c\right]\\ &=\frac{1}{2}\left(X_{(1)}+X_{(n)}\right)+c. \end{align*}

To see that neither the geometric nor the harmonic means are invariant to translations, we only need to find counterexamples. For that, consider the sample realization $$(x_1,x_2,x_3)=(1,2,3).$$ For these data, the geometric and harmonic means are, respectively,

\begin{align*} \left[\prod_{i=1}^n x_i\right]^{1/n}&=(1\times 2\times 3)^{1/3}=6^{1/3}=1.82,\\ \frac{n}{\sum_{i=1}^n x_i^{-1}}&=\frac{3}{1+\frac{1}{2}+\frac{1}{3}}=\frac{18}{11}=1.64. \end{align*}

However, if we take $$c=1$$:

\begin{align*} \left[\prod_{i=1}^n (x_i+c)\right]^{1/n}&=(2\times 3\times 4)^{1/3}=2.88\neq 1.82+1,\\ \frac{n}{\sum_{i=1}^n (x_i+c)^{-1}}&=\frac{3}{\frac{1}{2}+\frac{1}{3}+\frac{1}{4}}=\frac{36}{13}=2.77\neq 1.64+1, \end{align*}

and we see that none of these statistics is translation-invariant.

Example 3.9 A woman always arrives to the bus stop at the same hour. She wishes to estimate the maximum time waiting for the bus, knowing that the waiting time is distributed as $$\mathcal{U}(0,\theta).$$ For that purpose, she times the waiting times during $$n$$ days and obtains a realization of a srs, $$(x_1,\ldots,x_n),$$ measured in seconds. Based on that sample, she obtains an estimate of the maximum waiting time $$\hat{\theta}(x_1,\ldots,x_n)$$ in seconds. If she wants to convert the result to minutes, can she just compute $$\hat{\theta}(x_1,\ldots,x_n)/60$$?

The answer depends on whether the estimator satisfies

\begin{align*} \hat{\theta}(x_1/60,\ldots,x_n/60)=\hat{\theta}(x_1,\ldots,x_n)/60. \end{align*}

If this is not the case, then she will need to compute $$\hat{\theta}(x_1/60,\ldots,x_n/60).$$

Definition 3.4 (Scale-invariant estimators) An estimator $$\hat{\theta}$$ is scale-invariant if, for sample realization $$(x_1,\ldots,x_n)$$ and any $$c>0,$$

\begin{align*} \hat{\theta}(cx_1,\ldots,cx_n)=c\,\hat{\theta}(x_1,\ldots,x_n). \end{align*}

Example 3.10 Check that $$\bar{X}$$ and $$X_{(n)}$$ are scale-invariant estimators and that $$\log((1/n)\sum_{i=1}^n\exp({X_i}))$$ and $$X_{(n)}/X_{(1)}$$ are not.