4.7 Exact Binomial Test

The Clopper-Pearson exact binomial test is precise, but theoretically complicated in that it inverts two single-tailed binomial tests (No theory here - I’ll just rely on the software). Use the exact binomial test if you have a small sample size or an extreme success/failure probability that invalidates the chi-square and G tests. The exact binomial also applies when you have a one-tail test. The exact binomial test has two conditions:

independence, and
at least \(n\pi \ge 5\) successes or \(n(1−\pi)\ge 5\) failures.

You can use this test for multinomial variables too, but the test only compares a single level’s proportion to a hypothesized value.

Example

A pharmaceutical company claims its drug reduces fever in >60% of cases. In a random sample of n = 40 cases the drug reduces fever in 20 cases. Do you reject the claim?

You are testing \(P(x \le 20)\) in n = 40 trials when p = 60%, a one-tail test. The sample is a random assignment experiment with 20>5 successes and 20>5 failures, so it meets the conditions for the exact binomial test.

binom.test(20, 40, p = 0.6, alternative = "greater")

## 
##  Exact binomial test
## 
## data:  20 and 40
## number of successes = 20, number of trials = 40, p-value = 0.9256
## alternative hypothesis: true probability of success is greater than 0.6
## 95 percent confidence interval:
##  0.3610917 1.0000000
## sample estimates:
## probability of success 
##                    0.5

The exact binomial test uses the “method of small p-values”, in which the probability of observing a proportion \(p\) as far or further from \(\pi_0\) is the sum of all \(P(X=p_i)\) where \(p_i <= p\).

map_dbl(dbinom(0:20, 40, 0.6), ~if_else(. <= 0.5, ., 0)) %>% sum()

## [1] 0.1297657

That is what pbinom() does.

pbinom(q = 20, size = 40, p = 0.6, lower.tail = TRUE)

## [1] 0.1297657

A 95% confidence interval means 95% of confidence intervals constructed from a random sample of the population will contain the true population proportion. There are several methods to calculate a binomial confidence interval⁵ binom.test() uses the Clopper-Pearson interval. This method calculates lower (\(P_L\)) and upper (\(P_U\)) limits that satisfy

\[ \begin{eqnarray} \sum_{x=n_1}^n \binom{n}{x} p_L^x(1 - p_L)^{n-x} &=& \alpha/2\\ \sum_{x=0}^{n_1} \binom{n}{x} p_U^x(1 - p_U)^{n-x} &=& \alpha/2 \end{eqnarray} \]

where \(n_i\) is the measured successes in \(n\) trials. For a one-tail test, the confidence interval is calculated with the right side equaling 0 and \(/alpha\) instead of \(\alpha/2\). A right-tailed 95% confidence interval means 95% of confidence intervals will contain a lower limit that is less than the true population proportion. If you wanted to construct a confidence interval around the population proportion, use a two-sided test.

binom.test(20, 40, p = 0.6, alternative = "two.sided")

## 
##  Exact binomial test
## 
## data:  20 and 40
## number of successes = 20, number of trials = 40, p-value = 0.2007
## alternative hypothesis: true probability of success is not equal to 0.6
## 95 percent confidence interval:
##  0.3380178 0.6619822
## sample estimates:
## probability of success 
##                    0.5

If you just wanted to know whether 20 successes in 40 trials is compatible with a population proportion of 60%, then you could use the chi-squared goodness-of-fit test.

chisq.test(x = c(20, 20), p = c(0.6, 0.4), correct = FALSE)

## 
##  Chi-squared test for given probabilities
## 
## data:  c(20, 20)
## X-squared = 1.6667, df = 1, p-value = 0.1967

Wikipedia.↩︎