Section 11 Multivariate Analysis of Variance

Multivariate Analysis of Variance (MANOVA) is a technique that allows us to compare different populations. It is often used to determine whether the population mean vectors from different populations are the same (see Johnson, Wichern, and others (2014), Section 6.4, page 297).

Assumptions

In MANOVA we arrange the Random Samples obtained from l populations as follows:

\[\begin{align} Population_{1}:& X_{11},X_{12},...X_{1n_{1}}\\ Population_{2}:& X_{21},X_{22},...X_{2n_{2}}\\ \vdots & \\ Population_{l}:& X_{l1},X_{l2},...X_{ln_{l}} \end{align} \]

The key assumptions of a MANOVA Analysis become:

1 \(X_{g1},X_{g2},...X_{gn_{g}}\) is a Random Sample of size \(n_{g}\) from a population with mean \(\mu_{g}\),g=1,..,l. The Random Samples from different populations are independent.

2 All populations have a common covariance matrix \(\Sigma\)

3 Each population is distributed as a multivariate normal. This if often implied through studying mean vectors in the limit of large Samples (ie. Central Limit Theorem applied)

MANOVA Model

The jth (p-dimensional) observation belonging to the ith population is modelled according to the following equation:

\[X_{ij}=\mu+\tau_{i}+e_{ij}\]

where \(e_{ij}\) is an independent \(N_{p}(0,\sigma^2)\) multivariate normal random variable. The parameter vector \(\mu\) is an overall mean level and \(\tau_{i}\) is the treatment effect of the ith population. The parameter \(\mu\) is chosen so that \(\sum_{i=1}^{l}n_{i}\tau_{i}=0\).

Hypothesis Testing

The test statistic in MANOVA hypothesis tests is Wilks lamba \(\Lambda^*\). It is the quotient of the sum of squares and cross products between populations with the (mean corrected) total sum of squares.

\[\Lambda^*=\frac{|W|}{|B + W|}=\frac{|\sum_{i=1}^l\sum_{j=1}^{n_{l}}(x_{ij}-\overline{x}_{i})(x_{ij}-\overline{x}_{i})^´|}{|\sum_{i=1}^l\sum_{j=1}^{n_{l}}(x_{ij}-\overline{x})(x_{ij}-\overline{x})^´|}\] Under the Null Hypothesis of no treatment effect between populations (\(\tau_{1}=\tau_{2}=\tau_{3}=\dots=\tau_{l}=0\)), we are testing whether \(\Lambda^*\) is too small to reject. The exact distribution of \(\Lambda^*\) has been derived for special cases and is typically output with the result set in computer packages.

Bonferroni Confidence Statements

For pairwise comparisons, simultaneous (Bonferroni) confidence intervals for the components of differences between treatment means can be constructed (see Johnson, Wichern, and others (2014), Section 6.5, page 308):

Proposition 11.1 (Simulataneous Confidence Statements) Let \(n=\sum_{i=1}^{l}n_{i}\). For the multivariate one-way MANOVA model, with confidence at least \((1-\alpha)\), \(\tau_{ki}-\tau_{li}\) belongs to:

\[\overline{x}_{ki}-\overline{x}_{li} \pm t_{n-g}\Bigg(\frac{\alpha}{pg(g-1)}\Bigg)\sqrt{\frac{w_{ii}}{n-g}\bigg(\frac{1}{n_{k}}+\frac{1}{n_{l}}\bigg)}\]

for all components i=1,…,p and all differences l<k=1,….,g. By \(w_{ii}\) we denote the i-th diagonal element of \(W\), the residual matrix of square errors.

\(\square\)

References

Johnson, Richard Arnold, Dean W Wichern, and others. 2014. Applied Multivariate Statistical Analysis. Vol. 4. Prentice-Hall New Jersey.