7.7 Sampling distributions of $\hat{\mu}$ and $\hat{\Sigma}$

Consider the GWN model in matrix form (7.1). The estimator of the $N \times 1$ matrix $\mu$ is the sample mean vector (7.16) and the estimator of the $N \times N$ matrix $\Sigma$ is the sample covariance matrix (7.17). In this section we give the sampling distributions for the mean vector $\hat{\mu}$ and for the elements of the covariance matrix $\hat{\Sigma}$.

7.7.1 Sampling distribution of $\hat{\mu}$

The following proposition gives the sampling distribution of the $N \times 1$ vector $\hat{\mu}$.

Proposition 7.8 (Sampling distribution of $\hat{\mu}$) Under the GWN model and for large enough T, the sampling distribution for $\hat{\mu}$ is multivariate normal:

$$\begin{equation} \hat{\mu} \sim N\left(\mu, ~ \frac{1}{T}\hat{\Sigma}\right). \end{equation}$$

This result shows that the elements of the vector $\hat{\mu}$ are correlated, with the correlations being the same as the asset return correlations.

Example 3.9 (Joint distribution of $\hat{\mu}_1$ and $\hat{\mu}_2$)

Consider the bivariate case ($N=2$). Proposition 7.8 tells us the joint distribution between $\hat{\mu}_1$ and $\hat{\mu}_2$ is bivariate normal:

$$\begin{equation} \left( \begin{array}{c} \hat{\mu}_1 \\ \hat{\mu}_2 \end{array} \right) \sim N\left( \left(\begin{array}{c} \mu_1 \\ \mu_2 \end{array} \right), ~ \frac{1}{T} \left( \begin{array}{cc} \hat{\sigma}_1^2 & \hat{\sigma}_{12} \\ \hat{\sigma}_{12} & \hat{\sigma}_{2}^{2} \end{array} \right) \right). \end{equation}$$

Hence, $\widehat{\mathrm{cov}}(\hat{\mu}_1, \hat{\mu}_2) = \frac{1}{T}\hat{\sigma}_{12}$ and $\widehat{\mathrm{cor}}(\hat{\mu}_1, \hat{\mu}_2)=\hat{\rho}_{12}$.

The fact that the elements of $\hat{\mu}$ are correlated and the correlations are equal to the correlations between the corresponding assets is a useful result. For example, if assets 1 and 2 are strongly positively (negatively) correlated, then $\hat{\mu}_1$ and $\hat{\mu}_2$ are strongly positively (negatively) correlated. As a result, a positive (negative) estimate of $\hat{\mu}_1$ will tend to be associated with a positive (negative) estimate of $\hat{\mu}_2$.

$\blacksquare$

7.7.2 Sampling distribution of the elements of $\hat{\Sigma}$

We can also determine the sampling distribution for the vector of unique elements of $\hat{\Sigma}$. This requires a bit of new matrix notation because the joint distribution of a $N \times N$ symmetric matrix of random variables is defined as the joint distribution of the $N(N+1)/2$ elements of the matrix stacked into a $N(N+1)/2 \times 1$ vector.

Define $\hat{\theta}$ as the $N(N+1)/2 \times 1$ vector of unique elements of $\Sigma$. These elements can be extracted using the $\mathrm{vech}(\cdot)$ operator, which extracts the elements on and below the main diagonal of $\hat{\Sigma}$ column-wise and stacks them into a $N(N+1)/2$ vector. For example, if $N=3$ then $\hat{\theta}$ is the $6 \times 1$ vector

$$\begin{equation} \hat{\theta} = \mathrm{vech}(\hat{\Sigma}) = \mathrm{vech} \left(\begin{array}{ccc} \hat{\sigma}_1^2 & \hat{\sigma}_{12} & \hat{\sigma}_{13} \\ \hat{\sigma}_{12} & \hat{\sigma}_2^2 & \hat{\sigma}_{23} \\ \hat{\sigma}_{13} & \hat{\sigma}_{23} & \hat{\sigma}_{3}^2 \end{array} \right) = \left(\begin{array}{c} \hat{\sigma}_1^2 \\ \hat{\sigma}_{12}\\ \hat{\sigma}_{13}\\ \hat{\sigma}_2^2\\ \hat{\sigma}_{23}\\ \hat{\sigma}_{3}^2 \end{array} \right). \end{equation}$$

Proposition 7.9 (Sampling distribution of the unique elements of $\hat{\Sigma}$) Under the GWN model and for large enough T, the sampling distribution for the $N(N+1)/2 \times 1$ vector $\hat{\theta} = \mathrm{vech}(\hat{\Sigma})$ is multivariate normal:

$$\begin{equation} \hat{\theta} \sim N\left(\theta, ~ \frac{1}{T}\hat{\Omega}\right). \end{equation}$$

where $\frac{1}{T}\hat{\Omega}$ is an $N(N+1)/2 \times N(N+1)/2$ matrix giving the estimated variances and covariances of the unique elements of $\hat{\Sigma}$. The element of $\frac{1}{T}\hat{\Omega}$ corresponding to $\widehat{\mathrm{cov}}(\hat{\sigma}_{ij},\hat{\sigma}_{lm})$ is given by $T^{-1}(\hat{\sigma}_{il}\hat{\sigma}_{jm} + \hat{\sigma}_{im}\hat{\sigma}_{jl})$ for all $i,j,l,m = 1,2,\ldots,N$, including $i=j=l=m.$ If $i=j$ and $l=m$ then

$$\begin{align*} \widehat{\mathrm{cov}}(\hat{\sigma}_{ij},\hat{\sigma}_{lm}) & = \widehat{\mathrm{cov}}(\hat{\sigma}_{ii},\hat{\sigma}_{ll}) = \widehat{\mathrm{cov}}(\hat{\sigma}_i^2,\hat{\sigma}_l^2) \\ & =\frac{\hat{\sigma}_{il}\hat{\sigma}_{il} + \hat{\sigma}_{il}\hat{\sigma}_{il}}{T} = \frac{\hat{\sigma}_{il}^2 + \hat{\sigma}_{il}^2}{T} = \frac{2\hat{\sigma}_{il}^2}{T} \end{align*}$$ .

If $i=j=l=m$ then

$$\begin{align*} \widehat{\mathrm{cov}}(\hat{\sigma}_{ij},\hat{\sigma}_{lm}) & = \widehat{\mathrm{cov}}(\hat{\sigma}_i^2,\hat{\sigma}_i^2) = \widehat{\mathrm{var}}(\hat{\sigma}_i^2) \\ & =\frac{\hat{\sigma}_i^2\hat{\sigma}_i^2 + \hat{\sigma}_i^2\hat{\sigma}_i^2}{T} = \frac{2\hat{\sigma}_i^4}{T} \end{align*}$$ .

In Proposition 7.9, it is not easy to see what are the elements of $\hat{\Omega}$. A simple example can make things more clear. Consider the bivariate case $N=2$. Then the result in Proposition 7.9 gives:

$$\begin{align*} \hat{\theta} = \left(\begin{array}{c} \hat{\sigma}_1^2 \\ \hat{\sigma}_{12} \\ \hat{\sigma}_2^2 \end{array} \right) & \sim N\left( \left(\begin{array}{c} \sigma_1^2\\ \sigma_{12} \\ \sigma_2^2 \end{array} \right),~ \frac{1}{T}\left( \begin{array}{ccc} \hat{\omega}_{11} & \hat{\omega}_{12} & \hat{\omega}_{13} \\ \hat{\omega}_{21} & \hat{\omega}_{22} & \hat{\omega}_{23} \\ \hat{\omega}_{31} & \hat{\omega}_{32} & \hat{\omega}_{33} \end{array} \right) \right) \\ & \sim N\left( \left(\begin{array}{c} \sigma_1^2\\ \sigma_{12} \\ \sigma_2^2 \end{array} \right),~ \left( \begin{array}{ccc} \widehat{\mathrm{var}}(\hat{\sigma}_1^2) & \widehat{\mathrm{cov}}(\hat{\sigma}_1^2, \hat{\sigma}_{12}) & \widehat{\mathrm{cov}}(\hat{\sigma}_1^2, \hat{\sigma}_2^2) \\ \widehat{\mathrm{cov}}(\hat{\sigma}_{12}, \hat{\sigma}_1^2) & \widehat{\mathrm{var}}(\hat{\sigma}_{12}) & \widehat{\mathrm{cov}}(\hat{\sigma}_{12}, \hat{\sigma}_2^2) \\ \widehat{\mathrm{cov}}(\hat{\sigma}_1^2, \hat{\sigma}_2^2) & \widehat{\mathrm{cov}}(\hat{\sigma}_2^2, \hat{\sigma}_{12}) & \widehat{\mathrm{var}}(\hat{\sigma}_2^2) \end{array} \right) \right) \\ & \sim N\left( \left(\begin{array}{c} \sigma_1^2\\ \sigma_{12} \\ \sigma_2^2 \end{array} \right),~ \frac{1}{T}\left( \begin{array}{ccc} 2\hat{\sigma}_1^4 & 2\hat{\sigma}_1^2\hat{\sigma}_{12} & 2\hat{\sigma}_{12}^2 \\ 2\hat{\sigma}_1^2 \hat{\sigma}_{12} & \hat{\sigma}_1^2 \hat{\sigma}_2^2 + \hat{\sigma}_{12}^2 & 2\hat{\sigma}_{12}\hat{\sigma}_2^2 \\ 2\hat{\sigma}_{12} & 2\hat{\sigma}_{12} \hat{\sigma}_2^2 & 2\hat{\sigma}_2^4 \end{array} \right) \right). \end{align*}$$

Notice that the results for the diagonal elements of $\frac{1}{T}\hat{\Omega}$ match those given earlier (see equations (7.20) and (7.22)). What is new here are formulas for the off-diagonal elements.

7.7.3 Joint sampling distribution between $\hat{\mu}$ and $\mathrm{vech}(\hat{\Sigma})$

When we study hypothesis testing in Chapter 9 and the statistical analysis of portfolios in Chapter 15 we will need to use results on the joint sampling distribution between $\hat{\mu}$ and $\mathrm{vech}(\hat{\Sigma})$. The follow proposition states the result:

Proposition 7.10 (Joint sampling distribution between $\hat{\mu}$ and $\mathrm{vech}(\hat{\Sigma})$) Under the GWN model and for large enough T, the joint sampling distribution for the $N+N(N+1)/2 \times 1$ vector $(\hat{\mu}', \hat{\theta}')'$, where $\hat{\theta} = \mathrm{vech}(\hat{\Sigma})$ is multivariate normal $$\begin{equation} \left(\begin{array}{c} \hat{\mu} \\ \hat{\theta} \end{array} \right) \sim N \left( \left( \begin{array}{c} \mu \\ \theta \end{array} \right), ~ \frac{1}{T} \left( \begin{array}{cc} \hat{\Sigma} & 0 \\ 0 & \hat{\Omega} \end{array} \right) \right). \end{equation}$$

Proposition 7.10 tells us that the vectors $\hat{\mu}$ and $\hat{\theta}$ are jointly multivariate normally distributed and that all elements of $\hat{\mu}$ are independent of all the elements of $\hat{\theta}$.

Example 2.35 (Joint distribution of $(\mu,\sigma)'$)

Consider the GWN model for a single asset and suppose we are interested in the joint distribution of the vector $(\hat{\mu}, \hat{\sigma}^2)'$. Then Proposition 7.10 gives us the result:

$$\begin{equation} \left( \begin{array}{c} \hat{\mu} \\ \hat{\sigma}^2 \end{array} \right) \sim N \left( \left( \begin{array}{c} \mu \\ \sigma^2 \end{array} \right), ~ \left( \begin{array}{cc} \frac{\hat{\sigma}^2}{T} & 0 \\ 0 & \frac{2\hat{\sigma}^4}{T} \end{array} \right) \right). \tag{7.44} \end{equation}$$

Here, we see that $\mathrm{cov}(\hat{\mu}, \hat{\sigma}^2)=0$, which implies that $\hat{\mu}$ and $\hat{\sigma}^2$ are independent since they are jointly normally distributed. Furthermore, since $\hat{\mu}$ and $\hat{\sigma}^2$ are independent and function of $\hat{\mu}$ is independent of any function of $\hat{\sigma}^2$. For example, $\hat{\sigma} = \sqrt{\hat{\sigma}^2}$ is independent of $\hat{\mu}$. Using (7.34) we can then deduce the joint distribution of $(\hat{\mu}, \hat{\sigma}^2)'$:

$$\begin{equation} \left( \begin{array}{c} \hat{\mu} \\ \hat{\sigma} \end{array} \right) \sim N \left( \left( \begin{array}{c} \mu \\ \sigma \end{array} \right), ~ \left( \begin{array}{cc} \frac{\hat{\sigma}^2}{T} & 0 \\ 0 & \frac{\hat{\sigma}^2}{2T} \end{array} \right) \right). \tag{7.45} \end{equation}$$

$\blacksquare$