6.6 Power of a test and Neyman–Pearson’s Lemma

Definition 6.6 (Power function) The power function of a test φ is the function ω:Θ[0,1] that gives the probability of rejecting H0 from a srs (X1,,Xn) generated from F(;θ), for each θΘ:

ω(θ)=P(Reject H0|θ)=P(φ(X1,,Xn)=1|θ).

Remark. Recall that for θ=θ0Θ0 and H0:θ=θ0, the power equals the significance level:

ω(θ0)=P(Reject H0|θ0)=α.

In addition, for any value θ=θ1Θ1,

ω(θ1)=P(Reject H0|θ1)=1P(Do not reject H0|θ1)=1β(θ1),

that is, the power in θ1 equals the complementary of the Type II error probability for θ1 seen in Section 6.1.

Remark. Observe that the power function of the test φ is only informative on the expected behavior of φ; the function does not depend on the particular sample realization at hand.

Example 6.18 Consider (X1,,Xn) a srs of N(μ,σ2) with σ2 known, the test statistic

T(X1,,Xn)=ˉXμ0σ/n

and the hypothesis tests

  1. H0:μ=μ0 vs. H1:μ>μ0;
  2. H0:μ=μ0 vs. H1:μ<μ0;
  3. H0:μ=μ0 vs. H1:μμ0

with rejection regions

  1. Ca={Z>zα};
  2. Cb={Z<zα};
  3. Cc={|Z|>zα/2}.

Let us compute and plot the power functions for the testing problems a, b, and c.

In the first case:

ωa(μ)=P(T(X1,,Xn)>zα)=P(ˉXμ0σ/n>zα)=P(ˉXμ0+(μμ)σ/n>zα)=P(ˉXμσ/n>zαμμ0σ/n)=P(Z<zα+μμ0σ/n)=P(Z<zα+μμ0σ/n)=Φ(zα+μμ0σ/n),

where ZN(0,1). As expected, ωa(μ0)=Φ(zα)=α.

The second case is analogous:

ωb(μ)=P(T(X1,,Xn)<zα)=P(Z<zαμμ0σ/n)=Φ(zα+μ0μσ/n).

The third case follows now as a combination of the previous two ones:

ωc(μ)=P(|T(X1,,Xn)|>zα/2)=P(T(X1,,Xn)<zα/2)+P(T(X1,,Xn)>zα/2)=Φ(zα/2+μ0μσ/n)+Φ(zα/2+μμ0σ/n).

For fixed μ0=0, α=0.10, n=25, and σ=1 we can visualize the power functions μωa(μ), μωb(μ), and μωc(μ) in Figure 6.2. The curves give the probability of rejection of H0:μ=μ0 in favor of H1 using a sample of size n, as a function on what is the underlying truth μ.

Power functions \(\mu\mapsto\omega_a(\mu),\) \(\mu\mapsto\omega_b(\mu),\) and \(\mu\mapsto\omega_c(\mu)\) from Example 6.18 with \(\mu_0=0\) (dashed vertical line), \(\alpha=0.10\) (dotted horizontal line), \(n=25,\) and \(\sigma=1.\) Observe how a one-sided test has power below \(\alpha\) against alternatives in the opposite direction and how the two-sided test almost merges the powers of the two one-sided tests.

Figure 6.2: Power functions μωa(μ), μωb(μ), and μωc(μ) from Example 6.18 with μ0=0 (dashed vertical line), α=0.10 (dotted horizontal line), n=25, and σ=1. Observe how a one-sided test has power below α against alternatives in the opposite direction and how the two-sided test almost merges the powers of the two one-sided tests.

The usual criterion for selecting among several types of tests for the same hypothesis consists in fixing the Type I error probability, α, and then selecting among the tests with the same Type I error the one that presents the highest power for all θ1Θ1. This is the so-called Uniformly Most Powerful (UMP) test. However, the UMP test does not always exists.

The Neyman–Pearson’s Lemma guarantees the existence of the UMP test for the testing problem in which the null and the alternative hypotheses are simple, and provides us the form of the critical region for such a test. This region is based on a ratio of the likelihoods of the two parameter values appearing in H0 and H1.

Theorem 6.1 (Neyman–Pearson’s Lemma) Let XF(;θ). Assume it is desired to test

H0:θ=θ0vs.H1:θ=θ1

using the information of a srs (X1,,Xn) of X. For the significance level α, the test that maximizes the power in θ1 has a critical region of the form

C={(x1,,xn)Rn:L(θ0;x1,,xn)L(θ1;x1,,xn)<k}.

Recall that the Neyman–Pearson’s Lemma specifies only the form of the critical region, but not the specific value of k. However, k could be computed from the significance level α and the distribution of (X1,,Xn) under H0.

Example 6.19 Assume that X represents a single observation of the rv with pdf

f(x;θ)={θxθ1,0<x<1,0,otherwise.

Find the UMP test at a significance level α=0.05 for testing

H0:θ=1vs.H1:θ=2.

Let us employ Theorem 6.1. In this case we have a srs of size one. Then, the likelihood is

L(θ;x)=θxθ1.

For computing the critical region of the UMP test, we obtain

L(θ0;x)L(θ1;x)=L(1;x)L(2;x)=12x

and therefore

C={xR:12x<k}={x>12k}={x>k}.

The value of k can be determined from the significance level α,

α=P(Reject H0|H0 true)=P(x>k|θ=1)=1kf(x;1)dx=1kdx=1k.

Therefore, k=1α, so the critical region of UMP test of size α is

C={x:x>1α}.

When testing one-sided hypothesis of the type

H0:θ=θ0vs.H1:θ>θ0,

the Neyman–Pearson’s Lemma is not applicable.

However, if we fix a value θ1>θ0 and we compute the critical region of the UMP test for

H0:θ=θ0vs.H1:θ=θ1,

quite often the critical region obtained does not depend on the value θ1. Therefore, this very same test is the UMP test for testing (6.6)!

In addition, if we have an UMP test for

H0:θ=θ0vs.H1:θ>θ0,

then the same test is also the UMP test for

H0:θ=θ0vs.H1:θ>θ0,

since for any value θ0<θ0, any other test will have larger errors of the two types.