Chapter 3 Conditional Distributions

Nike and Adidas have opened up new neighboring sports stores which have found themselves in direct competition with each other. Let $S_{\text{Nike}}$ be the total sales in the Nike store, and let $S_{\text{Adidas}}$ be the total sales in the Adidas store. Phil Knight, the co-founder of Nike, would like to know the sales of the Adidas store in order to make business decisions regarding his own store. However Phil is not privy to this information. Phil only knows the exact sales of the Nike store. Given the information about $S_{\text{Nike}}$, what can be said about $S_{\text{Adidas}}$?

3.1 Conditional Probabilities and Discrete Conditional Distributions

The problem described above concerning sales in a Nike and Adidas store is in a continuous setting, that is, $S_1$ and $S_2$ are both continuous random variables. We have encountered this type of conditional problem in the context of probabilities. In this section, we recall this theory and extend it to discrete random variables.

Consider two events $A$ and $B$ such that $P(B)>0$. The probability of event $A$ happening given that $B$ has already happened is called the conditional probability of $A$ given $B$, and is denoted $P(A \mid B)$. The conditional probability can be calculated by the formula $$P(A \mid B) = \frac{P(A \cap B)}{P(B)}.$$

Why does Definition 3.1.1 specify that $P(B)>0$? The reason for this has both a real-world interpretation and a mathematical one. The statement that $P(B)=0$ is equivalent to $B$ being an impossible event. If $B$ was an impossible event, then the assumption that $B$ has already happened in Definition 3.1.1 would be a contradiction. Mathematically if $P(B)=0$, then we are unable to divide by $P(B)$ in the formula $P(A \mid B) = \frac{P(A \, \cap \, B)}{P(B)}$. Therefore we must specify that $P(B)>0$.

Consider the cafe from Example 2.5.2, where the owner has collected data to determine the joint probabilities of the temperature $X$ in degrees Celsius during winter and the number of customers $Y$ in the cafe each day. The joint probability table is

The owner would like to make a decision about how many staff are on shift today. Given that it is going to be $20^\circ C$, what is the probability that there are $75$ customers?

In mathematical language, the question is asking us to calculate $P(Y=75 \mid X=20)$. Using the formula of Definition 3.1.1, we know that $P(Y=75 \mid X=20) = \frac{P(X=20 \, \cap \, Y=75)}{P(X=20)}$. Calculate that

$$\begin{align*} P(X=20) &= P(X=20, Y=15) + P(X=20, Y=75) + P(X=20, Y=150) \\[3pt] &= 0.04 + 0.05 + 0.01 \\[3pt] &= 0.1 \end{align*}$$

Therefore

$$\begin{align*} P(Y=75 \mid X=20) &= \frac{P(X=20 \cap Y=75)}{P(X=20)} \\[3pt] &= \frac{0.05}{0.1} \\[3pt] &= \frac{1}{2} \end{align*}$$

Can we extend this idea of conditional probability to the language of discrete random variables?

Recall that for a random variable $X$, the probability $P(X=x)$ is given by the evaluation of the probability mass function $p_X(x)$ of $X$. Similarly for two random variables $X$ and $Y$, the probability $P(X=x, \, Y=y)$ is given by the evaluation of the joint probability mass function $p_{X,Y}(x,y)$. Converting Definition 3.1.1 into these terms leads us to the following.

Consider two discrete random variables $X$ and $Y$. Let $p_{X,Y}(x,y)$ be the joint PMF of $X$ and $Y$, and $p_Y(y)$ be the marginal PMF of $Y$. The conditional probability mass function of $X$ given that $Y=y$, denoted $p_{X|Y}(x \mid y)$, is the function

$$p_{X|Y}(x|y) = \begin{cases} \frac{p_{X,Y}(x,y)}{p_Y(y)}, & \text{if } p_Y(y)>0, \\[3pt] 0, & \text{if } p_Y(y)=0. \end{cases}$$

The conditional PMF $p_{Y|X}(y \mid x)$ is defined similarly.

The value $p_{X|Y}(x|y)$ is equal to the probability $P(X=x \mid Y=y)$.

Calculate the conditional PMF of $Y$ given $X$ for the random variables given in Example 3.1.2.

By applying Definition 3.1.3, calculate that

$$\begin{align*} p_{Y|X}(15|0) &= \frac{p_{X,Y}(15,0)}{p_X(0)} = \frac{0.07}{0.07 + 0.11 + 0.01} = 0.368, \\[5pt] p_{Y|X}(75|0) &= \frac{p_{X,Y}(75,0)}{p_X(0)} = \frac{0.11}{0.19} = 0.579, \\[5pt] p_{Y|X}(150|0) &= \frac{p_{X,Y}(150,0)}{p_X(0)} = \frac{0.01}{0.19} = 0.053, \\[5pt] p_{Y|X}(15|10) &= \frac{p_{X,Y}(15,10)}{p_X(10)} = \frac{0.23}{0.23 + 0.43 + 0.05} = 0.323, \\[5pt] p_{Y|X}(75|10) &= \frac{p_{X,Y}(75,10)}{p_X(10)} = \frac{0.43}{0.71} = 0.606, \\[5pt] p_{Y|X}(150|10) &= \frac{p_{X,Y}(150,10)}{p_X(10)} = \frac{0.05}{0.71} = 0.071, \\[5pt] p_{Y|X}(15|20) &= \frac{p_{X,Y}(15,20)}{p_X(20)} = \frac{0.04}{0.04 + 0.05 + 0.01} = 0.4, \\[5pt] p_{Y|X}(75|20) &= \frac{p_{X,Y}(75,20)}{p_X(20)} = \frac{0.05}{0.1} = 0.5, \\[5pt] p_{Y|X}(150|20) &= \frac{p_{X,Y}(150,20)}{p_X(20)} =\frac{0.01}{0.1} = 0.1. \end{align*}$$ Note that the calculation of $p_{Y|X}(75|20)$ is consistent with the solution of Example 3.1.2.

Let $y$ be some fixed outcome for the random variable $Y$. The sum of conditional probabilities $p_{X|Y}(x|y)$ over all values of $x$ will sum to $1$ since this is a complete set of possible outcomes. This can be verified for the solution to Example 3.1.4, for example:

$$p_{Y|X}(15|0) + p_{Y|X}(75|0) + p_{Y|X}(150|0) = 0.368 + 0.579 + 0.053 = 1.$$

Check that the sum, over all possible outcomes $x$, of values $p(x \mid 10)$ and values $p(x \mid 20)$ respectively, both equal $1$.

Consider two discrete random variables $X$ and $Y$. The conditional CDF of $X$ given $Y=y$ is $$F_{X|Y}(x|y) = \sum_{x' \leq x} p_{X|Y}(x'|y).$$

It follows from Definition 3.1.5 that $F_{X|Y}(x|y) = P(X \leq x|Y=y)$.

3.2 Continuous Conditional Distributions

Consider the Nike versus Adidas example at the opening of the chapter. This is a conditional probability problem but now in the continuous setting. For two continuous random variables $X$ and $Y$, the formula $P(Y=y \mid X=x) = \frac{P(X=x \, \cap \, Y=y)}{P(X=x)}$ breaks down because $P(X=x) = 0$ for a fixed value $x$ since $X$ is continuous.

The theory of discrete random variables in Section 3.1 motivates the definition of conditional random variables in the continuous setting.

Consider two continuous random variables $X$ and $Y$. Let $f_{X,Y}$ be the joint PDF of $X$ and $Y$, and $f_Y(y)$ be the marginal PDF of $Y$. The conditional PDF of $X$ given that $Y=y$ is defined by

$$f_{X|Y}(x|y) = \begin{cases} \frac{f_{X,Y}(x,y)}{f_Y(y)}, & \text{if } f_Y(y)>0, \\[5pt] 0, & \text{if } f_Y(y)=0. \end{cases}$$

This definition avoids the above problem that $f_Y(y) \neq P(Y=y)$.

Consider the two random variables $X,Y$ from Example 2.1.6 governing scores in a game played between Abbie and Bertie. The joint PDF is

$$f_{X,Y}(x,y) = \begin{cases} 24x(1-x-y), & \text{if } x,y \geq 0 \text{ and } x+y \leq 1, \\[5pt] 0, & \text{otherwise.} \end{cases}$$

Find the conditional PDF of Abbie’s score given that Bertie scored $\frac{1}{2}$.

In mathematical language, the question asks us to calculate $f_{X \mid Y}\left( x \mid \frac{1}{2} \right)$. In Example 2.1.8, we found

$$f_Y(y) = \begin{cases} 4(1-y)^3, & 0 \leq y \leq 1, \\[3pt] 0, & \text{otherwise.} \end{cases}$$ Therefore by Definition 3.1.4, $$\begin{align*} f_{X|Y}(x|y) &= \frac{f_{X,Y}(x,y)}{f_Y(y)} \\[5pt] &= \begin{cases} \frac{24x(1-x-y)}{4(1-y)^3}, & \text{if } x,y \geq 0 \text{ and } x+y \leq 1, \\[3pt] 0, & \text{otherwise.} \end{cases} \\[5pt] &= \begin{cases} \frac{6x(1-x-y)}{(1-y)^3}, & \text{if } x,y \geq 0 \text{ and } x+y \leq 1, \\[3pt] 0, & \text{otherwise.} \end{cases} \end{align*}$$ Setting $y=\frac{1}{2}$ obtain $$\begin{align*} f_{X|Y}\left(x \bigg| \frac{1}{2}\right) &= \frac{f_{X,Y}(x,\frac{1}{2})}{f_Y(\frac{1}{2})} \\[5pt] &= \begin{cases} \frac{6x \left( \frac{1}{2}-x \right)}{\left( \frac{1}{2} \right)^3}, & \text{if } 0 \leq x \leq \frac{1}{2}, \\[3pt] 0, & \text{otherwise,} \end{cases} \\[5pt] &= \begin{cases} 48x \left( \frac{1}{2}-x \right) , & \text{if } 0 \leq x \leq \frac{1}{2}, \\[3pt] 0, & \text{otherwise.} \end{cases} \end{align*}$$

Let $y$ be some fixed outcome for the random variable $Y$. It can be shown that $\int_{\mathbb{R}} \, f_{X \mid Y} (x \mid y) \,dx =1$. This is the continuous analogous to the result for discrete random variable that $\sum\limits_{x} f_{X \mid Y} (x \mid y) =1$ for a fixed value $y$.

Let $S$ be a subset of $\mathbb{R}$. What is the probability that the random variable $X$ belongs to $S$ given that $Y=y$? This can be calculated using the conditional PDF $f_{X|Y}(x|y)$:

$$P(X \in S \mid Y=y) = \int_S f_{X \mid Y}(x \mid y) \, dx.$$

This leads us to the following definition.

Consider two continuous random variables $X$ and $Y$. The conditional CDF of $X$ given $Y=y$ is

$$F_{X|Y}(x|y) = \int_{-\infty}^x f_{X|Y}(u|y) \,du.$$

It follows from Definition 3.2.3 that $F_{X|Y}(x|y) = P(X \leq x|Y=y)$.