22  Expected Values of Linear Combinations of Random Variables

22.1 Linear rescaling

If $X$ is a random variable and $a,b$ are non-random constants then

$$\begin{array}{rl}\text{E}(aX+b)& =a\text{E}(X)+b\\ \text{SD}(aX+b)& =|a|\text{SD}(X)\\ \text{Var}(aX+b)& =a^2\text{Var}(X)\end{array}$$

22.2 Linearity of expected value

Example 22.1 Refer to the tables and plots in Example 5.29 in the textbook. Each scenario contains SAT Math ($X$) and Reading ($Y$) scores for 10 hypothetical students, along with the total score ($T=X+Y$) and the difference between the Math and Reading scores ($D=X-Y$, negative values indicate lower Math than Reading scores). Note that the 10 $X$ values are the same in each scenario, and the 10 $Y$ values are the same in each scenario, but the $(X,Y)$ values are paired in different ways: the correlation is 0.78 in scenario 1, -0.02 in scenario 2, and -0.94 in scenario 3.

1. What is the mean of $T=X+Y$ in each scenario? How does it relate to the means of $X$ and $Y$? Does the correlation affect the mean of $T=X+Y$?
   
   
   

2. What is the mean of $D=X-Y$ in each scenario? How does it relate to the means of $X$ and $Y$? Does the correlation affect the mean of $D=X-Y$?
   
   
   

• Linearity of expected value. For any two random variables $X$ and $Y$, $$\begin{array}{rl}\text{E}(X+Y)& =\text{E}(X)+\text{E}(Y)\end{array}$$
• That is, the expected value of the sum is the sum of expected values, regardless of how the random variables are related.
• Therefore, you only need to know the marginal distributions of $X$ and $Y$ to find the expected value of their sum. (But keep in mind that the distribution of $X+Y$ will depend on the joint distribution of $X$ and $Y$.)
• Whether in the short run or the long run, $$\begin{array}{rl}\text{Average of }X+Y& =\text{Average of }X+\text{Average of }Y\end{array}$$ regardless of the joint distribution of $X$ and $Y$.
• A linear combination of two random variables $X$ and $Y$ is of the form $aX+bY$ where $a$ and $b$ are non-random constants. Combining properties of linear rescaling with linearity of expected value yields the expected value of a linear combination. $$\text{E}(aX+bY)=a\text{E}(X)+b\text{E}(Y)$$
• Linearity of expected value extends naturally to more than two random variables.

22.3 Variance of linear combinations of random variables

Example 22.2 Consider a random variable $X$ with $\text{Var}(X)=1$. What is $\text{Var}(2X)$?

• Walt says: $\text{SD}(2X)=2\text{SD}(X)$ so $\text{Var}(2X)=2^2\text{Var}(X)=4(1)=4$.
• Jesse says: Variance of a sum is a sum of variances, so $\text{Var}(2X)=\text{Var}(X+X)$ which is equal to $\text{Var}(X)+\text{Var}(X)=1+1=2$.

Who is correct? Why is the other wrong?

Example 22.3 Recall Example Example 22.1.

1. In which of the three scenarios is $\text{Var}(X+Y)$ the largest? Can you explain why?
   
   
   
   
   

2. In which of the three scenarios is $\text{Var}(X+Y)$ the smallest? Can you explain why?
   
   
   
   
   

3. In which scenario is $\text{Var}(X+Y)$ roughly equal to the sum of $\text{Var}(X)$ and $\text{Var}(Y)$?
   
   
   
   
   

4. In which of the three scenarios is $\text{Var}(X-Y)$ the largest? Can you explain why?
   
   
   
   
   

5. In which of the three scenarios is $\text{Var}(X-Y)$ the smallest? Can you explain why?
   
   
   
   
   

6. In which scenario is $\text{Var}(X-Y)$ roughly equal to the sum of $\text{Var}(X)$ and $\text{Var}(Y)$?
   
   
   
   
   

• Variance of sums and differences of random variables. $$\begin{array}{rl}\text{Var}(X+Y)& =\text{Var}(X)+\text{Var}(Y)+2\text{Cov}(X,Y)\\ \text{Var}(X-Y)& =\text{Var}(X)+\text{Var}(Y)-2\text{Cov}(X,Y)\end{array}$$

Example 22.4 Assume that SAT Math ($X$) and Reading ($Y$) scores follow a Bivariate Normal distribution, Math scores have mean 527 and standard deviation 107, and Reading scores have mean 533 and standard deviation 100. Compute $\text{Var}(X+Y)$ and $\text{SD}(X+Y)$ for each of the following correlations.

1. $\text{Corr}(X,Y)=0.77$
   
   
   
   
   

2. $\text{Corr}(X,Y)=0.40$
   
   
   
   
   

3. $\text{Corr}(X,Y)=0$
   
   
   
   
   

4. $\text{Corr}(X,Y)=-0.77$
   
   
   
   
   

Example 22.5 Continuing the previous example. Compute $\text{Var}(X-Y)$ and $\text{SD}(X-Y)$ for each of the following correlations.

1. $\text{Corr}(X,Y)=0.77$
   
   
   
   
   

2. $\text{Corr}(X,Y)=0.40$
   
   
   
   
   

3. $\text{Corr}(X,Y)=0$
   
   
   
   
   

4. $\text{Corr}(X,Y)=-0.77$
   
   
   
   
   

• The variance of the sum is the sum of the variances if and only if $X$ and $Y$ are uncorrelated. $$\begin{array}{rl}\text{Var}(X+Y)& =\text{Var}(X)+\text{Var}(Y)\text{if }X,Y\text{ are uncorrelated}\\ \text{Var}(X-Y)& =\text{Var}(X)+\text{Var}(Y)\text{if }X,Y\text{ are uncorrelated}\end{array}$$
• The variance of the difference of uncorrelated random variables is the sum of the variances
• If $a,b,c$ are non-random constants and $X$ and $Y$ are random variables then $$\text{Var}(aX+bY+c)=a^2\text{Var}(X)+b^2\text{Var}(Y)+2ab\text{Cov}(X,Y)$$

22.4 Bilinearity of covariance

$$\begin{array}{rl}\text{Cov}(X,X)& =\text{Var}(X)\\ \text{Cov}(X,Y)& =\text{Cov}(Y,X)\\ \text{Cov}(X,c)& =0\\ \text{Cov}(aX+b,cY+d)& =ac\text{Cov}(X,Y)\\ \text{Cov}(X+Y,U+V)& =\text{Cov}(X,U)+\text{Cov}(X,V)+\text{Cov}(Y,U)+\text{Cov}(Y,V)\end{array}$$