Section 6 Models Overview

The following sections on Multivariate Models are drawn from Johnson, Wichern, and others (2014). The proofs for results stated in the Linear Regression Section are found in Chapter 7, the proofs for the Principal Component Section are found in Chapters 8 & 8A, those for the Canonical Correlation Analysis in Chapter 10, those for Factor Analyis in Chapter 9.

Estimating Best Linear Predictors

In practice, the models we use to estimate best linear predictors make assumptions in addition to the population mean and covariance being finite. These assumptions vary from model to model. One practical reason is we do not have access to the population mean \(\mu\) and covariance matrix \(\Sigma\), but rather must make inferences based on sample estimates using the data available.

In the subsequent Data and Analysis sections, I want to investigate the best linear predictor of a response variable using four distinct Multivariate Techniques. My general objective is to end up with models with a limited number of factors (ie. data reduction) that offer significant insights into the data (ie ease of interpretation) and are technically sound (ie. statistical model fitting and selection).

As we shall see each approach emphasises a different aspect of the sample data. The Multivariate Analysis of Variance and Classical Linear Regression approaches focus on the sample mean. The Principal Component Analysis approach emphasises the sample covariance matrix and Factor Analysis is a mixture of both.

References

Johnson, Richard Arnold, Dean W Wichern, and others. 2014. Applied Multivariate Statistical Analysis. Vol. 4. Prentice-Hall New Jersey.