2.4 Testing Hypotheses

2.4.1 Null hypothesis vs. alternative hypothesis

https://en.wikipedia.org/wiki/Test_statistic

2.4.2 The Neyman-Pearson Lemma

Consider a hypothesis test with null hypothesis $H_0$ and alternative hypothesis $H_1$. Let $X$ be a random variable with probability density function (or probability mass function) $f(x; \theta_0)$ under $H_0$ and $f(x; \theta_1)$ under $H_1$. Let $T$ be a test statistic, and let $C$ be the critical region (rejection region) of the test. The Neyman-Pearson Lemma states that the most powerful test of size $\alpha$ is given by the likelihood ratio test, which rejects $H_0$ if

$$\frac{f(x; \theta_1)}{f(x; \theta_0)} > k$$

where $k$ is a constant chosen such that the probability of type I error is equal to $\alpha$, i.e.,

$$P(X \in C | \theta = \theta_0) = \alpha.$$

Equivalently, the most powerful test can also be expressed as rejecting $H_0$ if

$$\log \frac{f(x; \theta_1)}{f(x; \theta_0)} > \log k = c$$

where $c$ is a constant.

If the likelihood ratio is such that $\frac{f(x; \theta_1)}{f(x; \theta_0)} = k$ on a set of probability zero under $H_0$ and $H_1$, then the test is still most powerful. If the likelihood ratio is always constant, then no test is more powerful than any other.