## 4.1 Definitions

The mean (or the expected value) of a rv, say $$X$$, is denoted $$E[X]$$ (and sometimes by $$\mu$$) and is defined as $E[X] = \begin{cases} \displaystyle \int_S x f(x)\, dx & \qquad\text{for x continuous};\notag\\ ~&\\ \displaystyle E[X] = \sum_S x f(x) & \qquad\text{for x discrete}.\notag\\ \end{cases}$

where $$S$$ is the sample space. The mean is also called the first moment about $$0$$.

More generally, the expected value of any function of $$X$$, say $$g(X)$$, is denoted $$E[g(X)]$$ and is defined as $E[g(X)] = \begin{cases} \displaystyle\int_S g(x) f(x)\, dx & \qquad\text{for x continuous};\notag\\ ~&\\ \displaystyle\sum_S g(x) f(x) & \qquad\text{for x discrete}.\notag \end{cases}$ One important application of this is for the variance, where $$g(X) = (x-\mu)^2$$. The variance of a rv, say $$X$$, is denoted $$\text{var}[X]$$ and is defined as $\text{var}[X] = E[(x-\mu)^2] = \begin{cases} \displaystyle\int_S (x-\mu)^2 f(x)\, dx &\qquad\text{for x continuous};\notag\\ ~&\\ \displaystyle\sum_S (x-\mu)^2 f(x) &\qquad\text{for x discrete}.\notag \end{cases}$ The variance is called the second moment about the mean. The standard deviation is the positive square root of the variance, and measures the amount of variation in the rv $$X$$.

The covariance between two variables, say $$X$$ and $$Z$$, is \begin{align} \text{Cov}[X, Z] &= E[ (X-E[X]) (Z-E[Z])]\notag\\ &= E[XZ] - E[X]E[Z].\notag \end{align} If $$\text{Cov}[X, Z] = 0$$, then $$X$$ and $$Z$$ are said to be uncorrelated.

If $$X$$ and $$Z$$ are independent, then $$\text{Cov}[X, Z] = 0$$.

However, if $$\text{Cov}[X,Z] = 0$$ then $$X$$ and $$Z$$ may be or may not be independent.