Chapter 10 Probability

Students tend to find these two chapters on probability the most difficult material in the course. It looks hard, but it’s really not as complicated as it looks. It is important to understand how probabilities work in order to reason well. We rarely have conclusive evidence for or against any claim. Imagine that you’re on a jury trial, you have been tasked with determining the probability of the defendant’s guilt given the evidence. To do this well requires that a person have a basic understanding of how probability works.

Given the example about serving on a jury, it’s more than a little disturbing that our intuitions about probabilities are extremely flawed. Here’s a classic example called the Monty Hall Problem: In the game show Let’s Make a Deal, the host, Monty Hall, would select a person to play for the big prize. The contestant would have a choice of three doors. After choosing a door, the host, who knows which door the prize is behind, would open one of the other doors and show the contestant that that door did not reveal the prize. The contestant would then be offered the choice to stick with his original choice or to switch to the third door.

So, you are the contestant. You choose door number 1. Let’s say that Monty opens door 2 and shows you that it has nothing behind it. What should you do? Stick with 1 or switch to 3? You should do what will increase the probability of your winning. Which has the higher probability? Most people will answer that, since there are only two doors, neither has a higher probability than the other. So, the common answer goes, the odds of your winning are simply 50/50.

The correct answer, though, is that you should switch. If you switch, the probability of winning doubles. Is this intuitive? Absolutely not.

10.1 Calculating Probabilities

First, a few preliminaries. Probabilities are numbers between 0 and 1. Unfortunately, it will be necessary to be able to add, multiply, and divide fractions. If you can’t remember how, look at the review in section 13.6 of the text.

We won’t worry about the morality of gambling, but it’s easiest to learn basic probability in the context of cards, dice, and coin tosses. Basic probability questions are often about cards and dice. So, a few facts to keep in mind:

  1. Each die has six sides.
  2. A standard deck of cards has 52 cards, divided into 4 suits (clubs, diamonds, hearts, and spades). Each suit has an Ace, 2, 3, 4, 5, 6, 7, 8, 9, 10, Jack, Queen, and King. We’ll never have Jokers in our imaginary decks.
  3. \(\Pr(S) = n\) means that the probability that the sentence S is true is equal to n.
  4. Probabilities range from zero to one, inclusive. The answer to a probability problem will never be less than zero nor greater than one.

10.2 Calculating Simple Events

Examples of simple events include tossing heads with one toss of a single coin, getting a six with a roll of a single die, and drawing a heart on one draw from a standard deck of cards. The rule for calculating the probability of a simple event is this:

\[ \Pr(A)=\frac{\textrm{favorable outcomes}}{\textrm{total possible outcomes}} \]

That’s easy enough. We just have to determine how many possible ways this scenario could work out, and how many of those ways get us the outcome that we’re looking for.

10.2.1 Examples

What is the probability of tossing heads with a single coin? If we toss a coin once, there are only two possible outcomes (to keep things simple, we rule out the very, very slim possibility that it lands and stays on edge). Of those two outcomes. only one is heads. So, the probability of tossing heads is equal to 1/2. In probability notation, \(\Pr(H) = 1/2\)

What is the probability of rolling two on one roll with a single die? There are six possible outcomes, only one is a two. \(\Pr(2) = 1/6\)

What is the probability of drawing the Ace of Spades on one draw from a deck of cards? There is one favorable outcome out of 52 total possible: \(\Pr(A\spadesuit) = 1/52\)

What is the probability of drawing an Ace on a single draw? Now there are four favorable outcomes in the 52 total possible: \(\Pr(A) = 4/52 = 1/13\)

What is the probability of drawing a Heart on a single draw? Since there are thirteen Hearts, there are 13 favorable outcomes, but still only 52 cards. \(\Pr(A) = 13/52 = 1/4\)

10.3 Calculating Complex Events

Complex probabilities are probabilities of negations, conjunctions, or disjunctions. A negation is a “not” sentence. The sentence “I will not go to the movies tonight” is the negation of the sentence “I will go to the movies tonight.” A conjunction is an “and” sentence. An example is the sentence “I will go to dinner and I will go to the movies.” A disjunction is an “or” sentence, as in “I will go to dinner or I will go to the movies.” Unless specified otherwise, disjunctions are always inclusive disjunctions. So, “I will go to dinner or I will go to the movies” means that I will do one, or the other, or both.

There are also some symbols that you need to know. “Not” is symbolized by “\(\neg\)” and “and” is symbolized by “&”

  1. The probability of \(\neg P\) is the probability that P is not true.
  2. The probability of \(P\&Q\) is the probability that both P and Q are true.
  3. The probablity of \(P \textrm{ or } Q\) the probability that either P or Q or both are true.

10.3.1 Necessities and Impossibilities

\[ \textrm{If S cannot be true, then} \Pr(S) = 0 \]

\[ \textrm{If S must be true, then} \Pr(S) = 1 \]

What is the probability of rolling a 7 with one die? Since it is impossible to roll more than a six with one die, \(\Pr(7) = 0\).

What is the probability of rolling at least a 1 with one die? No matter what you roll, you will get at least a 1, so \(\Pr (\textrm{at least 1}) = 1\).

10.3.2 Negations

Now, let’s pause and think for a moment. Remember that the probability of an event that must occur is equal to one. For any event, it must be the case that some outcome occurs. For example, if you toss a coin, you have to get either heads or tails. So, if you add up the probabilities of all the possible outcomes for an event, they have to add up to 1. Now, let’s imagine an event that has three possible outcomes, A, B, and C. By our reasoning, \(\Pr(A) + \Pr(B) + \Pr(C) = 1\). That is, we can think of the probability of an event as represented by a big pie. Each possible outcome is a piece of the pie. The size of the whole pie is 1, so when we add up the areas of each the pieces, they have to total 1. Now, let’s I want to know the probability of \(\neg A\). The probability of A is just the size of A in the pie. The probability of \(\neg A\) then is the size of the remainder of the pie, once we take out A. Since the size of all of the pieces add up to 1, the probability of A not occuring is equal to \(1 - \Pr(A)\). Thus, we get the negation rule:

\[ \Pr(\neg S) = 1-\Pr(S) \] For example, what is the probability of not rolling a 6 on one roll of a die? It must be equal to 1 minus the probability of rolling a six. Since there a six sides, the probability of rolling a six equals 1/6. So, \(\Pr(\neg 6) = 1 - 1/6 = 5/6\)

What is the probability of not drawing the King of Hearts? \(1 - Pr(K\heartsuit) = 1 − 1/52 = 51/52\)

10.3.3 Compatibility

Two statements are compatible if they can both be true, and two events are compatible if they can both occur. There are certain events such that one’s occurring automatically rules out the other’s occurring. For example, if I get heads on one coin toss, that means that I didn’t get tails. There’s no way for both of those to occur on the same toss. They are incompatible events. Other events are compatible. Let’s say I draw one card from a deck. Can I get both a King and a Heart? Yes, if I get the King of Hearts. So, getting a King and getting a Heart on the same draw are compatible events.

So, compatible or incompatible?

  1. Tossing heads on one coin toss and tossing tails on the same toss. Incompatible
  2. Tossing heads on one toss and tossing tails on the next. Compatible.
  3. Drawing the ace of spades on both of two draws, if
  4. The first card is put back into the deck (with replacement). Compatible
  5. The first card is not put back into the deck (without replacement). Incompatible

10.3.4 Incompatible Disjunctions

If A and B are incompatible, then \(\Pr(A \textrm{ or } B) = \Pr(A) + \Pr(B)\)

What is the probability of getting either heads or tails on one coin toss? The two events are incompatible, you can’t get both, so \(\Pr(H \textrm{ or } T) = \Pr(H) + \Pr(T) = 1/2 + 1/2 = 1\). Of course it equals one, since you must get one or the other.

What is the probability of getting either a king or a queen on one draw from a deck? They are incompatible, you can’t get both, so \(\Pr(K \textrm{ or } Q) = \Pr(K) + \Pr(Q) = 1/13 +1/13 = 2/13\).

What is the probability of getting the Ace of Spades or a heart on one draw? Incompatible, so \(\Pr(A\spadesuit \textrm{ or } \heartsuit) = \Pr(A\spadesuit) + \Pr(\heartsuit) = 1/52 + 13/52 = 14/52 = 7/26\).

10.3.5 Compatible Disjunctions

What about compatible events? First, let’s see why that formula will not work. What is the probability of getting heads at least once on two coin tosses? If we use the formula for incompatible events, we have \(\Pr (H1 \textrm{ or } H2) = \Pr(H1) + \Pr (H2) = 1/2 + 1/2 = 1\). This cannot be right! Why not? If it has a probability of one, it must occur, but we know it’s possible to get tails on both tosses. In fact, we know the answer should be 3/4. (I’ll let you figure out why. Think in terms of favorable and total possible.)

So what went wrong with the formula? Essentially, we counted the same thing twice. If we toss a coin twice, there are four possible outcomes: HH, HT, TH, and TT. Out of those, there are two ways to get H on toss 1 and two ways to get H on toss 2. Adding those, it looks like we have four favorable outcomes. The problem is that we have counted one of those favorable outcomes (HH) twice, so we need to #tract one of them.

That gives us the rule for compatible disjunctions:

\[ \textrm{If A and B are compatible, then } \Pr (A \textrm{ or } B) = \Pr(A) + \Pr(B) − \Pr(A \& B) \]

The problem now is that \(\Pr(A \& B)\) is the probability of a conjunction. So, before we can calculate compatible disjunctions, we need to learn to calculate the probabilities of conjunctions.

10.3.6 Independent Conjunctions

To calculate disjunctions, we have to determine whether they are compatible or incompatible. For conjunctions, we’re concerned about dependence and independence. Two statements are independent if the truth value of one has no bearing on the truth value of the other. Two events are independent if the occurrence of one has no bearing on the truth value of the other.

For example, tossing heads on one toss and tossing tails on the same toss are dependent events. If the first happens, the probability of the second becomes zero. Tossing heads on one toss and then tossing tails on the next toss are independent events. he second has a probability of 1/2 whether the first occurs or not. Card draws are independent if you put the cards back as you draw them (called with replacement). Card draws are dependent if the cards are not placed back in the deck (without replacement).

\[ \textrm{If A and B are independent, then } \Pr(A \& B) = \Pr(A) \times \Pr(B) \]

Here are some examples. What is the probability of getting heads on two consecutive tosses? That means getting heads on the first toss and heads on the second toss. The two are independent events, so \(\Pr(H1 \& H2) = \Pr(H1) \times \Pr(H2) = 1/2 \times 1/2 = 1/4\).

What’s the probability of getting a king on two consecutive draws with replacement? Again, these are independent events. \(\Pr(K1 \& K2) = \Pr(K1) \times \Pr(K2) =1/13 \times 1/13 = 1/169\).