3.5 Positive Definite Matrices

Let \(\mathbf{A}\) be an \(n\times n\) symmetrix matrix. The matrix \(\mathbf{A}\) is positive definite if for any \(n\times1\) vector \(\mathbf{x\neq0}\) \[ \mathbf{x}^{\prime}\mathbf{A}\mathbf{x}>0. \] This condition implies that there is no non-zero vector \(\mathbf{x}\) such that \(\mathbf{Ax}=\mathbf{0}\), which implies that \(\mathbf{A}\) has full rank \(n\).

The matrix \(\mathbf{A}\) is positive semi-definite if for any \(n\times1\) vector \(\mathbf{x\neq0}\) \[ \mathbf{x}^{\prime}\mathbf{A}\mathbf{x}\geq0. \] Hence, if \(\mathbf{A}\) is positive semi-definite then there exists some \(n\times1\) vector \(\mathbf{x}\) such that \(\mathbf{Ax}=\mathbf{0}\), which implies that \(\mathbf{A}\) does not have full rank \(n\).

3.5.1 Matrix square root

To motivate the idea of the square root of a matrix, consider the square root factorization of a positive number \(a:\) \[ a=\sqrt{a}\times\sqrt{a}=a^{1/2}\times a^{1/2}. \] That is, we can factorize any positive number into the product of its square root. It turns out we can do a similar factorization for a positive definite and symmetric matrix.

Let \(\mathbf{A}\) be an \(n\times n\) positive definite and symmetric matrix. Then it is possible to factor \(\mathbf{A}\) as \[ \mathbf{A}=\mathbf{C^{\prime}}\mathbf{C}, \] where \(\mathbf{C}\) is an \(n\times n\) upper triangular matrix with non-negative diagonal elements called the Cholesky factor of \(\mathbf{A}\). This factorization is also called a matrix square root factorization by defining the square root matrix \(\mathbf{A}^{1/2}=\mathbf{C}\). Then we can write \(\mathbf{A}=\mathbf{A}^{1/2\prime}\mathbf{A}^{1/2}\).¹⁶ If all of the diagonal elements of \(\mathbf{C}\) are positive then \(\mathbf{A}\) is positive definite. Otherwise, \(\mathbf{A}\) is positive semi-definite.

Example 2.24 (Cholesky decomposition in R)

The R function chol() computes the Cholesky factorization of a square symmetrix matrix. For example, consider the \(2\times2\) correlation matrix:

Rho = matrix(c(1, 0.75, 0.75, 1), 2, 2)
Rho

##      [,1] [,2]
## [1,] 1.00 0.75
## [2,] 0.75 1.00

The Cholesky matrix square root factorization is

C = chol(Rho)
C

##      [,1]      [,2]
## [1,]    1 0.7500000
## [2,]    0 0.6614378

t(C)%*%C

##      [,1] [,2]
## [1,] 1.00 0.75
## [2,] 0.75 1.00

Here, all the diagonal elements of \(\mathbf{C}\) are positive so that covariance matrix is positive definite. Next, suppose the correlation matrix is

Rho = matrix(c(1, 1, 1, 1), 2, 2)
Rho

##      [,1] [,2]
## [1,]    1    1
## [2,]    1    1

so that the two variables are perfectly correlated. Then

chol(Rho, pivot=TRUE)

## Warning in chol.default(Rho, pivot = TRUE): the matrix is either rank-deficient
## or indefinite

##      [,1] [,2]
## [1,]    1    1
## [2,]    0    0
## attr(,"pivot")
## [1] 1 2
## attr(,"rank")
## [1] 1

Here, we see that one of the diagonal elements of the Cholesky factor is zero and that the correlation matrix is rank deficient.

\(\blacksquare\)

---

16. The Cholesky factorization is one way of defining the matrix square root, where the square root matrix is upper triangular. Other matrix square factorizations exist where the square root matrix is not necessarily upper triangular.↩︎