Chapter 11 Odds and Bayes Factors

Example 11.1 The ELISA test for HIV was widely used in the mid-1990s for screening blood donations. As with most medical diagnostic tests, the ELISA test is not perfect. If a person actually carries the HIV virus, experts estimate that this test gives a positive result 97.7% of the time. (This number is called the sensitivity of the test.) If a person does not carry the HIV virus, ELISA gives a negative (correct) result 92.6% of the time (the specificity of the test). Estimates at the time were that 0.5% of the American public carried the HIV virus (the base rate).

Suppose that a randomly selected American tests positive; we are interested in the conditional probability that the person actually carries the virus.

  1. Before proceeding, make a guess for the probability in question. \[ 0-20\% \qquad 20-40\% \qquad 40-60\% \qquad 60-80\% \qquad 80-100\% \]
  2. Denote the probabilities provided in the setup using proper notation
  3. Construct an appropriate two-way table and use it to compute the probability of interest.
  4. Construct a Bayes table and use it to compute the probability of interest.
  5. Explain why this probability is small, compared to the sensitivity and specificity.
  6. By what factor has the probability of carrying HIV increased, given a positive test result, as compared to before the test?
\iffalse{} Solution. to Example 11.1
Show/hide solution

  1. We don’t know what you guessed, but from experience many people guess 80-100%. Afterall, the test is correct for most of people who carry HIV, and also correct for most people who don’t carry HIV, so it seems like the test is correct most of the time. But this argument ignores one important piece of information that has a huge impact on the results: most people do not carry HIV.

  2. Let \(H\) denote the event that the person carries HIV (hypothesis), and let \(E\) denote the event that the test is positive (evidence). Therefore, \(H^c\) is the event that the person does not carry HIV, another hypothesis. We are given

    • prior probability: \(P(H) = 0.005\)
    • likelihood of testing positive, if the person carries HIV: \(P(E|H) = 0.977\)
    • \(P(E^c|H^c) = 0.926\)
    • likelihood of testing positive, if the person does not carry HIV: \(P(E|H^c) = 1-P(E^c|H^c) = 1-0.926 = 0.074\)
    • We want to find the posterior probability \(P(H|E)\).

  3. Considering a hypothetical population of Americans (at the time)

    • 0.5% of Americans carry HIV
    • 97.7% of Americans who carry HIV test positive
    • 92.6% of Americans who do not carry HIV test negative
    • We want to find the percentage of Americans who test positive that carry HIV.

  4. Assuming 1000000 Americans

    Tests positive Does not test positive Total
    Carries HIV 4885 115 5000
    Does not carry HIV 73630 921370 995000
    Total 78515 921485 1000000

    Among the 78515 who test positive, 4885 carry HIV, so the probability that an American who tests positive actually carries HIV is 4885/78515 = 0.062.

  5. See the Bayes table below.

  6. The result says that only 6.2% of Americans who test positive actually carry HIV. It is true that the test is correct for most Americans with HIV (4885 out of 5000) and incorrect only for a small proportion of Americans who do not carry HIV (73630 out of 995000). But since so few Americans carry HIV, the sheer number of false positives (73630) swamps the number of true positives (4885).

  7. Prior to observing the test result, the prior probability that an American carries HIV is \(P(H) = 0.005\). The posterior probability that an American carries HIV given a positive test result is \(P(H|E)=0.062\). \[ \frac{P(H|E)}{P(H)} = \frac{0.062}{0.005} = 12.44 \] An American who tests positive is about 12.4 times more likely to carry HIV than an American whom the test result is not known. So while 0.067 is still small in absolute terms, the posterior probability is much larger relative to the prior probability.

hypothesis prior likelihood product posterior
Carries HIV 0.005 0.977 0.0049 0.0622
Does not carry HIV 0.995 0.074 0.0736 0.9378
sum 1.000 NA 0.0785 1.0000

Remember, the conditional probability of \(H\) given \(E\), \(P(E|H)\), is not the same as the conditional probability of \(E\) given \(H\), \(P(E|H)\), and they can be vastly different. It is helpful to think of probabilities as percentages and ask “percent of what?” For example, the percentage of people who carry HIV that test positive is a very different quantity than the percentage of people who test positive that carry HIV. Make sure to properly identify the “denominator” or baseline group the percentages apply to.

Posterior probabilities can be highly influenced by the original prior probabilities, sometimes called the base rates. The example illustrates that when the base rate for a condition is very low and the test for the condition is less than perfect there will be a relatively high probability that a positive test is a false positive. Don’t neglect the base rates when evaluating posterior probabilities

Example 11.2 True story: On a camping trip in 2003, my wife and I were driving in Vermont when, suddenly, a very large, hairy, black animal lumbered across the road in front of us and into the woods on the other side. It happened very quickly, and at first I said “It’s a gorilla!” But then after some thought, and much derision from my wife, I said “it was probably a bear.”

I think this story provides an anecdote about Bayesian reasoning, albeit bad reasoning at first but then good. Put the story in a Bayesian context by identifying hypotheses, evidence, prior, and likelihood. What was the mistake I made initially?

Show/hide solution
  • “Type of animal” is playing the role of the hypothesis: gorilla, bear, dog, squirrel, rabbit, etc.
  • That the animal is very large, hairy, and black is the evidence.
  • The likelihood value for the animal being very large, hairy, and black is close to 1 for both a bear and gorilla, maybe more middling for a dog, but close to 0 for a squirrel, rabbit, etc.

The mistake I made initially was to neglect the base rates and not consider my prior probabilities. Let’s say the likelihood is 1 for both gorilla and bear and 0 for all other animals. Then based solely on the likelihoods, the posterior probability would be 50/50 for gorilla and bear, which maybe is why I guessed gorilla.

After my initial reaction, I paused to formulate my prior probabilities, which considering I was in Vermont, gave much higher probability to a bear than a gorilla. (My prior probabilities should also have given even higher probability to animals such as dogs, squirrels, and rabbits.)

By combining prior and likelihood in the appropriate way, the posterior probability is

  • very high for a bear, due to high likelihood and not-too-small prior,
  • close to 0 for a gorilla, due to the very small prior,
  • and very low for a squirrel or rabbit or other small animals because of the close-to-zero likelihood, even if the prior is large.

Recall that the odds of an event is a ratio involving the probability that the event occurs and the probability that the event does not occur \[ \text{odds}(A) = \frac{P(A)}{P(A^c)} = \frac{P(A)}{1-P(A)} \] In many situations (e.g. gambling) odds are reported as odds against \(A\), that is, the odds in favor of \(A\) not occuring, a.k.a., the odds of \(A^c\): \(P(A^c)/P(A)\).

The probability of an even can be obtained from odds \[ P(A) = \frac{\text{odds}(A)}{1+\text{odds}(A)} \]

Example 11.3 Continuing Example 11.1

  1. In symbols and words, what does one minus the answer to the probability in question in Example 11.1 represent?
  2. Calculate the prior odds of a randomly selected American having the HIV virus, before taking an ELISA test.
  3. Calculate the posterior odds of a randomly selected American having the HIV virus, given a positive test result.
  4. By what factor has the odds of carrying HIV increased, given a positive test result, as compared to before the test? This is called the Bayes factor.
  5. Suppose you were given the prior odds and the Bayes factor. How could you compute the posterior odds?
  6. Compute the ratio of the likelihoods of testing positive, for those who carry HIV and for those who do not carry HIV. What do you notice?

\iffalse{} Solution. to Example 11.3

Show/hide solution
  1. \(1-P(H|E) = P(H^c|E)=0.938\) is the posterior probability that an American who has a positive test does not carry HIV.
  2. The prior probability of carrying HIV is \(P(H)=0.005\) and the prior probability of not carrying HIV is \(P(H^c) = 1-0.005 = 0.995\) \[ \frac{P(H)}{P(H^c)} = \frac{0.005}{0.995} = \frac{1}{199} \approx 0.005025 \] These are the prior odds in favor of carrying HIV. The prior odds against carrying HIV are \[ \frac{P(H^c)}{P(H)} = \frac{0.995}{0.005} = 199 \] That is, prior to taking the test, an American is 199 times more likely to not carry HIV than to carry HIV.
  3. The posterior probability of carrying HIV given a positive test is \(P(H|E)=0.062\) and the posterior probability of not carrying HIV given a positive test is \(P(H^c|E) = 1-0.062 = 0.938\). \[ \frac{P(H|E)}{P(H^c|E)} = \frac{0.062}{0.938} \approx 0.066 \] These are the posterior odds in favor of carrying HIV given a positive test. The posterior odds against carrying HIV given a positive test are \[ \frac{P(H^c|E)}{P(H|E)} = \frac{0.938}{0.062} \approx 15.1 \] That is, given a positive test, an American is 15.1 times more likely to not carry HIV than to carry HIV.
  4. Comparing the prior and posterior odds in favor of carrying HIV, \[ BF = \frac{\text{posterior odds}}{\text{prior odds}} = \frac{0.066}{0.005025} = 13.2 \] The odds of carrying HIV are 13.2 times greater given a positive test result than prior to taking the test. The Bayes Factor is \(BF = 13.2\).
  5. By definition \[ BF = \frac{\text{posterior odds}}{\text{prior odds}} \] Rearranging yields \[ \text{posterior odds} = \text{prior odds}\times BF \]
  6. The likelihood of testing positive given HIV is \(P(E|H) = 0.977\) and the likelihood of testing positive given no HIV is \(P(E|H^c) = 1-0.926 = 0.074\). \[ \frac{P(E|H)}{P(E|H^c)} = \frac{0.977}{0.074} = 13.2 \] This value is the Bayes factor! So we could have computed the Bayes factor without first computing the posterior probabilities or odds.
  • If \(P(H)\) is the prior probability of \(H\), the prior odds (in favor) of \(H\) are \(P(H)/P(H^c)\)
  • If \(P(H|E)\) is the posterior probability of \(H\) given \(E\), the posterior odds (in favor) of \(H\) given \(E\) are \(P(H|E)/P(H^c|E)\)
  • The Bayes factor (BF) is defined to be the ratio of the posterior odds to the prior odds \[ BF = \frac{\text{posterior odds}}{\text{prior odds}} = \frac{P(H|E)/P(H^c|E)}{P(H)/P(H^c)} \]
  • The odds form of Bayes rule says \[\begin{align*} \text{posterior odds} & = \text{prior odds} \times \text{Bayes factor}\\ \frac{P(H|E)}{P(H^c|E)} & = \frac{P(H)}{P(H^c)} \times BF \end{align*}\]
  • Apply Bayes rule to \(P(H|E)\) and \(P(H^c|E)\) \[\begin{align*} \frac{P(H|E)}{P(H^c|E)} & = \frac{P(E|H)P(H)/P(E)}{P(E|H^c)P(H^c)/P(E)}\\ & = \frac{P(H)}{P(H^c)} \times \frac{P(E|H)}{P(E|H^c)}\\ \text{posterior odds} & = \text{prior odds} \times \frac{P(E|H)}{P(E|H^c)} \end{align*}\]
  • Therefore, the Bayes factor for hypothesis \(H\) given evidence \(E\) can be calculated as the ratio of the likelihoods \[ BF = \frac{P(E|H)}{P(E|H^c)} \]
  • That is, the Bayes factor can be computed without first computing posterior probabilities or odds.
  • Odds form of Bayes rule \[\begin{align*} \frac{P(H|E)}{P(H^c|E)} & = \frac{P(H)}{P(H^c)} \times \frac{P(E|H)}{P(E|H^c)} \\ \text{posterior odds} & = \text{prior odds} \times \text{Bayes factor} \end{align*}\]

Example 11.4 Continuing Example 11.1. Now suppose that 5% of individuals in a high-risk group carry the HIV virus. Consider a randomly selectd person from this group who takes the test. Suppose the sensitivity and specificity of the test are the same as in Example 11.1.

  1. Compute and interpret the prior odds that a person carries HIV.
  2. Use the odds form of Bayes rule to compute the posterior odds that the person carries HIV given a positive test, and interpret the posterior odds.
  3. Use the posterior odds to compute the posterior probability that the person carries HIV given a positive test.

\iffalse{} Solution. to Example 11.4

  1. \(P(H)/P(H^c) = 0.05/0.95 = 1/19 \approx 0.0526\). A person in this group is 19 times more likely to not carry HIV than to carry HIV.
  2. The posterior odds are the product of the prior odds and the Bayes factor. The Bayes factor is the ratio of the likelihoods. Since the sensitivity and specificity are the same as in the previous example, the likelihoods are the same, and the Bayes factor is the same. \[ \frac{P(E|H)}{P(E|H^c)} = \frac{0.977}{0.074} = 13.2 \] Therefore \[ \text{posterior odds} = \text{prior odds} \times \text{Bayes factor} = \frac{1}{19} \times 13.2 \approx \frac{1}{1.44} \approx 0.695 \] Given a positive test, a person in this group is 1.44 times more likely to not carry HIV than to carry HIV.
  3. The odds is the ratios of the posterior probabilities, and we basically just rescale so they add to 1. The posterior probability is \[ P(H|E) = \frac{0.695}{1 + 0.695} = \frac{1}{1 + 1.44} \approx 0.410 \] The Bayes table is below; we have added a row for the ratios to illustrate the odds calculations.
hypothesis prior likelihood product posterior
Carries HIV 0.0500 0.977 0.0489 0.4100
Does not carry HIV 0.9500 0.074 0.0703 0.5900
sum 1.0000 NA 0.1191 1.0000
ratio 0.0526 13.203 0.6949 0.6949