30  Joint Normal Distributions

• Jointly continuous random variables $X$ and $Y$ have a Bivariate Normal distribution with parameters ${\mu }_X$, ${\mu }_Y$, ${\sigma }_X>0$, ${\sigma }_Y>0$, and $-1<\rho <1$ if the joint pdf is { $$\begin{array}{rl}{f}_{X,Y}(x,y)& =\frac{1}{2\pi {\sigma }_X{\sigma }_Y\sqrt{1-{\rho }^2}}\exp (-\frac{1}{2(1-{\rho }^2)}[{\left(\frac{x-{\mu }_X}{{\sigma }_X}\right)}^2+{\left(\frac{y-{\mu }_Y}{{\sigma }_Y}\right)}^2-2\rho \left(\frac{x-{\mu }_X}{{\sigma }_X}\right)\left(\frac{y-{\mu }_Y}{{\sigma }_Y}\right)]),-\mathrm{\infty }<x<\mathrm{\infty },-\mathrm{\infty }<y<\mathrm{\infty }\end{array}$$ }
• It can be shown that if the pair $(X,Y)$ has a BivariateNormal(${\mu }_X$, ${\mu }_Y$, ${\sigma }_X$, ${\sigma }_Y$, $\rho$) distribution $$\begin{array}{rl}\text{E}(X)& ={\mu }_X\\ \text{E}(Y)& ={\mu }_Y\\ \text{SD}(X)& ={\sigma }_X\\ \text{SD}(Y)& ={\sigma }_Y\\ \text{Corr}(X,Y)& =\rho \end{array}$$
• A Bivariate Normal Density has elliptical contours. For each height $c>0$ the set $\{(x,y):f_{X,Y}(x,y)=c\}$ is an ellipse. The density decreases as $(x,y)$ moves away from $({\mu }_X,{\mu }_Y)$, most steeply along the minor axis of the ellipse, and least steeply along the major of the ellipse.
• A scatterplot of $(x,y)$ pairs generated from a Bivariate Normal distribution will have a rough linear association and the cloud of points will resemble an ellipse.
• If $X$ and $Y$ have a Bivariate Normal distribution, then the marginal distributions are also Normal: $X$ has a Normal$({\mu }_X,{\sigma }_X)$ distribution and $Y$ has a Normal$({\mu }_Y,{\sigma }_Y)$.
• If $X$ and $Y$ have a Bivariate Normal distribution and $\text{Corr}(X,Y)=0$ then $X$ and $Y$ are independent. (Remember, in general it is possible to have situations where the correlation is 0 but the random variables are not independent.)
• It can also be shown that if $X$ and $Y$ have a Bivariate Normal distribution then any conditional distribution is Normal. The conditional distribution of $Y$ given $X=x$ is

$$N({\mu }_Y+\frac{\rho {\sigma }_Y}{{\sigma }_X}(x-{\mu }_X),{\sigma }_Y\sqrt{1-{\rho }^2})$$

• The conditional expected value of $Y$ given $X=x$ is a linear function of $x$, called the regression line of $Y$ on $X$: $$\text{E}(Y|X=x)={\mu }_Y+\rho {\sigma }_Y\left(\frac{x-{\mu }_X}{{\sigma }_X}\right)$$
  • The regression line passes through the point of means $({\mu }_X,{\mu }_Y)$ and has slope $$\frac{\rho {\sigma }_Y}{{\sigma }_X}$$
  • The regression line estimates that if the given $x$ value is $z$ SDs above of the mean of $X$, then the corresponding $Y$ values will be, on average, $\rho z$ SDs away from the mean of $Y$ $$\frac{\text{E}(Y|X=x)-{\mu }_Y}{{\sigma }_Y}=\rho \left(\frac{x-{\mu }_X}{{\sigma }_X}\right)$$
  • Since $|\rho |\le 1$, for a given $x$ value the corresponding $Y$ values will be, on average, relatively closer to the mean of $Y$ than the given $x$ value is to the mean of $X$. This is known as regression to the mean.
• For Bivariate Normal distributions, the conditional variance of $Y$ given $X=x$ does not depend on $x$: $$\text{SD}(Y|X=x)={\sigma }_Y\sqrt{1-{\rho }^2}$$
• $X$ and $Y$ have a Bivariate Normal distribution if and only if every linear combination of $X$ and $Y$ has a Normal distribution. That is, $X$ and $Y$ have a Bivariate Normal distribution if and only if $aX+bY+c$ has a Normal distribution for all $a$, $b$, $c$.

Example 30.1

Suppose that SAT Math ($M$) and Reading ($R$) scores of CalPoly students have a Bivariate Normal distribution. Math scores have mean 640 and SD 80, Reading scores have mean 610 and SD 70, and the correlation between scores is 0.7.

1. Find the probability that a student has a Math score above 700.
   
   
   
   
   
2. Find the probability that a student has a total score above 1500.
   
   
   
   
   
3. Compute and interpret $\text{E}(M|R=700)$.
   
   
   
   
   
4. Find the probability that a student has a higher Math than Reading score if the student scores 700 on Reading.
   
   
   
   
   
5. Describe how you could use a Normal(0, 1) spinner to simulate an $(X,Y)$ pair.
   
   
   
   
   
6. Find the probability that a student has a higher Math than Reading score.
   
   
   
   
   

Example 30.2

Let $X$ and $I$ be independent, $X$ has a Normal(0,1) distribution, and $I$ takes values 1 or $-1$ with probability $1/2$ each. Let $Y=IX$.

1. How could you use spinners to simulate an $(X,Y)$ pair?
   
   
   
   
   
2. Identify the distribution of $Y$.
   
   
   
   
   
3. Sketch a scatterplot of simulated $(X,Y)$ values.
   
   
   
   
   
4. Are $X$ and $Y$ independent? (Careful, it is not enough to say “no, because $Y$ is a function of $X$”. You can check that $Y$ and $I$ are independent even though $Y$ is a function of $I$.)
   
   
   
   
   
5. Find $\text{Cov}(X,Y)$ and $\text{Corr}(X,Y)$.
   
   
   
   
   
6. Is the distribution of $X+Y$ Normal? (Hint: find $\text{P}(X+Y=0)$.)
   
   
   
   
   
7. Does the pair $(X,Y)$ have a Bivariate Normal distribution?
   
   
   
   
   

• If the pair $(X,Y)$ has a joint Normal distribution then each of $X$ and $Y$ has a Normal distribution.
• But the example shows that the converse is not true. That is, if each of $X$ and $Y$ has a Normal distribution, it is not necessarily true that the pair $(X,Y)$ has a joint Normal distribution
• However, if $X$ and $Y$ are independent and each of $X$ and $Y$ has a Normal distribution, then the pair $(X,Y)$ has a joint Normal distribution.