4.1 Definitions

The mean (or the expected value) of a rv, say $X$, is denoted $E[X]$ (and sometimes by $\mu$) and is defined as \[ E[X] = \begin{cases} \displaystyle \int_S x f(x)\, dx & \qquad\text{for $x$ continuous};\notag\\ ~&\\ \displaystyle E[X] = \sum_S x f(x) & \qquad\text{for $x$ discrete}.\notag\\ \end{cases} \]

where $S$ is the sample space. The mean is also called the first moment about $0$.

More generally, the expected value of any function of $X$, say $g(X)$, is denoted $E[g(X)]$ and is defined as \[ E[g(X)] = \begin{cases} \displaystyle\int_S g(x) f(x)\, dx & \qquad\text{for $x$ continuous};\notag\\ ~&\\ \displaystyle\sum_S g(x) f(x) & \qquad\text{for $x$ discrete}.\notag \end{cases} \] One important application of this is for the variance, where $g(X) = (x-\mu)^2$. The variance of a rv, say $X$, is denoted $\text{var}[X]$ and is defined as \[ \text{var}[X] = E[(x-\mu)^2] = \begin{cases} \displaystyle\int_S (x-\mu)^2 f(x)\, dx &\qquad\text{for $x$ continuous};\notag\\ ~&\\ \displaystyle\sum_S (x-\mu)^2 f(x) &\qquad\text{for $x$ discrete}.\notag \end{cases} \] The variance is called the second moment about the mean. The standard deviation is the positive square root of the variance, and measures the amount of variation in the rv $X$.

The covariance between two variables, say $X$ and $Z$, is \[\begin{align} \text{Cov}[X, Z] &= E[ (X-E[X]) (Z-E[Z])]\notag\\ &= E[XZ] - E[X]E[Z].\notag \end{align}\] If $\text{Cov}[X, Z] = 0$, then $X$ and $Z$ are said to be uncorrelated.

If $X$ and $Z$ are independent, then $\text{Cov}[X, Z] = 0$.

However, if $\text{Cov}[X,Z] = 0$ then $X$ and $Z$ may be or may not be independent.