2  Measuring Uncertainty

2.1 What is Probability?

Probability is a measurement of how strongly we believe things about the world. We can consider probability an extension of logic. Probability allows us to extend logic not only to work with absolute beliefs (true and false) but also to work with uncertain values (values between true and false).

An important part of logic is negation. When we say “not true” we mean false. Likewise, saying “not false” means true. We want probability to work the same way, so we make sure that the probability of X and the negation of the probability of X sum to 1 (in other words, values are either X, or not X).

\[P(X) + \neg{P(X)} = 1 \tag{2.1}\]

2.2 Calculating Probabilities by Counting Outcomes of Events

The most common way to calculate probability is to count outcomes of events. We have two sets of outcomes that are important:

1. All possible outcomes of an event. For a coin toss, this would be “heads” or “tails.”
2. The count of the outcomes you’re interested in. If you’ve decided that heads means you win, the outcomes you care about are those involving heads.

In probability theory, we use \(\Omega\) (the capital Greek letter omega) to indicate the set of all events:

\[\Omega = \{heads, tails\} \tag{2.2}\]

Example 2.1 What is the probability of getting at least one heads when we toss two coins?

1. Count the list of all possible events:

\[\Omega = \{(heads, heads), (heads, tails), (tails, heads), (tails, tails) \} \tag{2.3}\]

2. How many events match our condition?

\[\{(heads, heads), (heads, tails), (tails, heads) \} \tag{2.4}\]

3. Divide number of matches by number of all possible events:

\[\frac{3}{4}\] Solving harder probability problems of this nature often involves a field of mathematics called combinatorics.

2.3 Calculating Probabilities as Ratios of Beliefs

Counting events is only useful for physical objects. How to reason about more abstract problems? – Making bets is a practical way that we can express how strongly we hold our beliefs.

2.3.1 Using Odds to Determine Probability

Odds are a common way to represent beliefs as a ratio of how much you would be willing to pay if you were wrong about the outcome of an event to how much you’d want to receive for being correct. For example, say the odds of a horse winning a race are 12 to 1.

Odds have a simple relation with probabilities: the odds of an outcome are the ratio of the probability that the outcome occurs to the probability that the outcome does not occur. In mathematical terms, where p is the probability of the outcome and 1- p is the probability that the outcome does not occur:

\[odds = \frac{p}{1-p} \tag{2.5}\]

2.3.2 Solving for the Probabilities

Using algebra we can convert Equation 2.5 to find the probability of a hypotheses:

\[p = \frac{odds}{1 + odds} \tag{2.6}\]

2.3.3 Measuring Beliefs in a Coin Toss

Rather than thinking about a coin toss as an event, we can rephrase the question as “How strongly do I believe the next coin toss will be heads?” Now we’re not talking about \(P(heads)\) but rather a hypothesis or belief about the coin toss, \(P(H_{heads})\). Just like before, we need an alternate hypothesis to compare our belief with.

2.4 Wrapping Up

• We explored two different types of probabilities: those of events and those of beliefs.
• We defined probability as the ratio of the outcome(s) we care about to the number of all possible outcomes.

While this is the most common definition of probability, it is difficult to apply to beliefs because most practical, everyday probability problems do not have clear-cut outcomes and so aren’t intuitively assigned discrete numbers. To calculate the probability of beliefs, then, we need to establish how many times more we believe in one hypothesis over another. One good test of this is how much you would be willing to bet on your belief

2.5 Exercises

Try answering the following questions to make sure you understand how we can assign real values between 0 and 1 to our beliefs.

Example 2.2 What is the probability of rolling two six-sided dice and getting a value greater than 7?

Following Example 2.1:

1. Equation 2.3: \(36\)

\[ \begin{align*} \Omega = \{(1,1), (1,2), (1,3), (1,4), (1,5), (1,6) \\ (2,1), (2,2), (2,3), (2,4), (2,5), (2,6) \\ (3,1), (3,2), (3,3), (3,4), (3,5), (3,6) \\ (4,1), (4,2), (4,3), (4,4), (4,5), (4,6) \\ (5,1), (5,2), (5,3), (5,4), (5,5), (5,6) \\ (6,1), (6,2), (6,3), (6,4), (6,5), (6,6) \} \end{align*} \] 2. Equation 2.4: \(15\)

Solution:

\[\frac{15}{36} = \frac{5}{12}\]

---

Example 2.3 What is the probability of rolling three six-sided dice and getting a value greater than 7?

Solution:

possible_events = dice_sum = success = 0
for (i in 1:6) {
    for (j in 1:6) {
        for (k in 1:6) {
            possible_events <- possible_events + 1
            dice_sum <- i + j + k
            if (dice_sum > 7) {
                success <- success + 1
            }
        }
    }
}

glue::glue('Possible events = {possible_events}, \n',
            'Dice value > 7 = {success}, \n',
           'Probability = {success} / {possible_events} = {success / possible_events}')

#> Possible events = 216, 
#> Dice value > 7 = 181, 
#> Probability = 181 / 216 = 0.837962962962963

The Yankees are playing the Red Sox. You’re a diehard Sox fan and bet your friend they’ll win the game. You’ll pay your friend $30 if the Sox lose and your friend will have to pay you only $5 if the Sox win. What is the probability you have intuitively assigned to the belief that the Red Sox will win?

Solution:

\[odds = \frac{30}{5} = 6\] Probability is calculated with Equation 2.6:

\[p = \frac{odds}{1 + odds} = \]

\(p = \frac{6}{1 + 6} = \frac{6}{7}\) = 0.86