6.6 Power of a test and Neyman–Pearson’s Lemma
Definition 6.6 (Power function) The power function of a test φ is the function ω:Θ→[0,1] that gives the probability of rejecting H0 from a srs (X1,…,Xn) generated from F(⋅;θ), for each θ∈Θ:
ω(θ)=P(Reject H0|θ)=P(φ(X1,…,Xn)=1|θ).
Remark. Recall that for θ=θ0∈Θ0 and H0:θ=θ0, the power equals the significance level:
ω(θ0)=P(Reject H0|θ0)=α.
In addition, for any value θ=θ1∈Θ1,
ω(θ1)=P(Reject H0|θ1)=1−P(Do not reject H0|θ1)=1−β(θ1),
that is, the power in θ1 equals the complementary of the Type II error probability for θ1 seen in Section 6.1.
Remark. Observe that the power function of the test φ is only informative on the expected behavior of φ; the function does not depend on the particular sample realization at hand.
Example 6.18 Consider (X1,…,Xn) a srs of N(μ,σ2) with σ2 known, the test statistic
T(X1,…,Xn)=ˉX−μ0σ/√n
and the hypothesis tests
- H0:μ=μ0 vs. H1:μ>μ0;
- H0:μ=μ0 vs. H1:μ<μ0;
- H0:μ=μ0 vs. H1:μ≠μ0
with rejection regions
- Ca={Z>zα};
- Cb={Z<−zα};
- Cc={|Z|>zα/2}.
Let us compute and plot the power functions for the testing problems a, b, and c.
In the first case:
ωa(μ)=P(T(X1,…,Xn)>zα)=P(ˉX−μ0σ/√n>zα)=P(ˉX−μ0+(μ−μ)σ/√n>zα)=P(ˉX−μσ/√n>zα−μ−μ0σ/√n)=P(−Z<−zα+μ−μ0σ/√n)=P(Z<−zα+μ−μ0σ/√n)=Φ(−zα+μ−μ0σ/√n),
where Z∼N(0,1). As expected, ωa(μ0)=Φ(−zα)=α.
The second case is analogous:
ωb(μ)=P(T(X1,…,Xn)<−zα)=P(Z<−zα−μ−μ0σ/√n)=Φ(−zα+μ0−μσ/√n).
The third case follows now as a combination of the previous two ones:
ωc(μ)=P(|T(X1,…,Xn)|>zα/2)=P(T(X1,…,Xn)<−zα/2)+P(T(X1,…,Xn)>zα/2)=Φ(−zα/2+μ0−μσ/√n)+Φ(−zα/2+μ−μ0σ/√n).
For fixed μ0=0, α=0.10, n=25, and σ=1 we can visualize the power functions μ↦ωa(μ), μ↦ωb(μ), and μ↦ωc(μ) in Figure 6.2. The curves give the probability of rejection of H0:μ=μ0 in favor of H1 using a sample of size n, as a function on what is the underlying truth μ.
![Power functions \(\mu\mapsto\omega_a(\mu),\) \(\mu\mapsto\omega_b(\mu),\) and \(\mu\mapsto\omega_c(\mu)\) from Example 6.18 with \(\mu_0=0\) (dashed vertical line), \(\alpha=0.10\) (dotted horizontal line), \(n=25,\) and \(\sigma=1.\) Observe how a one-sided test has power below \(\alpha\) against alternatives in the opposite direction and how the two-sided test almost merges the powers of the two one-sided tests.](inference_files/figure-html/pow-1.png)
Figure 6.2: Power functions μ↦ωa(μ), μ↦ωb(μ), and μ↦ωc(μ) from Example 6.18 with μ0=0 (dashed vertical line), α=0.10 (dotted horizontal line), n=25, and σ=1. Observe how a one-sided test has power below α against alternatives in the opposite direction and how the two-sided test almost merges the powers of the two one-sided tests.
The usual criterion for selecting among several types of tests for the same hypothesis consists in fixing the Type I error probability, α, and then selecting among the tests with the same Type I error the one that presents the highest power for all θ1∈Θ1. This is the so-called Uniformly Most Powerful (UMP) test. However, the UMP test does not always exists.
The Neyman–Pearson’s Lemma guarantees the existence of the UMP test for the testing problem in which the null and the alternative hypotheses are simple, and provides us the form of the critical region for such a test. This region is based on a ratio of the likelihoods of the two parameter values appearing in H0 and H1.
Theorem 6.1 (Neyman–Pearson’s Lemma) Let X∼F(⋅;θ). Assume it is desired to test
H0:θ=θ0vs.H1:θ=θ1
using the information of a srs (X1,…,Xn) of X. For the significance level α, the test that maximizes the power in θ1 has a critical region of the form
C={(x1,…,xn)′∈Rn:L(θ0;x1,…,xn)L(θ1;x1,…,xn)<k}.
Recall that the Neyman–Pearson’s Lemma specifies only the form of the critical region, but not the specific value of k. However, k could be computed from the significance level α and the distribution of (X1,…,Xn) under H0.
Example 6.19 Assume that X represents a single observation of the rv with pdf
f(x;θ)={θxθ−1,0<x<1,0,otherwise.
Find the UMP test at a significance level α=0.05 for testing
H0:θ=1vs.H1:θ=2.
Let us employ Theorem 6.1. In this case we have a srs of size one. Then, the likelihood is
L(θ;x)=θxθ−1.
For computing the critical region of the UMP test, we obtain
L(θ0;x)L(θ1;x)=L(1;x)L(2;x)=12x
and therefore
C={x∈R:12x<k}={x>12k}={x>k′}.
The value of k′ can be determined from the significance level α,
α=P(Reject H0|H0 true)=P(x>k′|θ=1)=∫1k′f(x;1)dx=∫1k′dx=1−k′.
Therefore, k′=1−α, so the critical region of UMP test of size α is
C={x:x>1−α}.
When testing one-sided hypothesis of the type
H0:θ=θ0vs.H1:θ>θ0,
the Neyman–Pearson’s Lemma is not applicable.
However, if we fix a value θ1>θ0 and we compute the critical region of the UMP test for
H0:θ=θ0vs.H1:θ=θ1,
quite often the critical region obtained does not depend on the value θ1. Therefore, this very same test is the UMP test for testing (6.6)!
In addition, if we have an UMP test for
H0:θ=θ0vs.H1:θ>θ0,
then the same test is also the UMP test for
H0:θ=θ′0vs.H1:θ>θ0,
since for any value θ′0<θ0, any other test will have larger errors of the two types.