## 3.2 Basic Matrix Operations

In this section we review the basic matrix operations of addition, subtraction, scalar multiplication and multiplication.

### 3.2.1 Addition and subtraction

Matrix addition and subtraction are element by element operations and only apply to matrices of the same dimension. For example, let, $\mathbf{A}=\left[\begin{array}{cc} 4 & 9\\ 2 & 1 \end{array}\right],~\mathbf{B}=\left[\begin{array}{cc} 2 & 0\\ 0 & 7 \end{array}\right].$ Then, \begin{align*} \mathbf{A+B} & =\left[\begin{array}{cc} 4 & 9\\ 2 & 1 \end{array}\right]+\left[\begin{array}{cc} 2 & 0\\ 0 & 7 \end{array}\right]=\left[\begin{array}{cc} 4+2 & 9+0\\ 2+0 & 1+7 \end{array}\right]=\left[\begin{array}{cc} 6 & 9\\ 2 & 8 \end{array}\right],\\ \mathbf{A-B} & =\left[\begin{array}{cc} 4 & 9\\ 2 & 1 \end{array}\right]-\left[\begin{array}{cc} 2 & 0\\ 0 & 7 \end{array}\right]=\left[\begin{array}{cc} 4-2 & 9-0\\ 2-0 & 1-7 \end{array}\right]=\left[\begin{array}{cc} 2 & 9\\ 2 & -6 \end{array}\right]. \end{align*}

Example 2.11 (Matrix addition and subtraction in R)

Matrix addition and subtraction is straightforward in R:

matA = matrix(c(4,9,2,1),2,2,byrow=TRUE)
matB = matrix(c(2,0,0,7),2,2,byrow=TRUE)
matA
##      [,1] [,2]
## [1,]    4    9
## [2,]    2    1
matB
##      [,1] [,2]
## [1,]    2    0
## [2,]    0    7
# matrix addition
matC = matA + matB
matC
##      [,1] [,2]
## [1,]    6    9
## [2,]    2    8
# matrix subtraction
matD = matA - matB
matD
##      [,1] [,2]
## [1,]    2    9
## [2,]    2   -6

$$\blacksquare$$

### 3.2.2 Scalar multiplication

Here we refer to the multiplication of a matrix by a scalar number. This is also an element-by-element operation. For example, let $$c=2$$ and, $\mathbf{A}=\left[\begin{array}{cc} 3 & -1\\ 0 & 5 \end{array}\right].$ Then, $c\cdot\mathbf{A}=\left[\begin{array}{cc} 2\cdot3 & 2\cdot(-1)\\ 2\cdot(0) & 2\cdot5 \end{array}\right]=\left[\begin{array}{cc} 6 & -2\\ 0 & 10 \end{array}\right].$

Example 3.3 (Scalar multiplication in R)
matA = matrix(c(3,-1,0,5),2,2,byrow=TRUE)
matC = 2*matA
matC
##      [,1] [,2]
## [1,]    6   -2
## [2,]    0   10

$$\blacksquare$$

### 3.2.3 Matrix multiplication

Matrix multiplication only applies to conformable matrices. $$\mathbf{A}$$ and $$\mathbf{B}$$ are conformable matrices if the number of columns in $$\mathbf{A}$$ is equal to the number of rows in $$\mathbf{B}.$$ For example, if $$\mathbf{A}$$ is $$n\times m$$ and $$\mathbf{B}$$ is $$m\times p$$ then $$\mathbf{A}$$ and $$\mathbf{B}$$ are conformable and the matrix product of $$\mathbf{A}$$ and $$\mathbf{B}$$ has dimension $$n\times p$$. The mechanics of matrix multiplication is best explained by example. Let, $\underset{(2\times2)}{\mathbf{A}}=\left[\begin{array}{cc} 1 & 2\\ 3 & 4 \end{array}\right]\textrm{ and}\underset{(2\times3)}{\mathbf{B}}=\left[\begin{array}{ccc} 1 & 2 & 1\\ 3 & 4 & 2 \end{array}\right].$ Then, \begin{align*} \underset{(2\times2)}{\mathbf{A}}\cdot\underset{(2\times3)}{\mathbf{B}} & =\left[\begin{array}{cc} 1 & 2\\ 3 & 4 \end{array}\right]\cdot\left[\begin{array}{ccc} 1 & 2 & 1\\ 3 & 4 & 2 \end{array}\right]\\ & =\left[\begin{array}{ccc} 1\cdot1+2\cdot3 & 1\cdot2+2\cdot4 & 1\cdot1+2\cdot2\\ 3\cdot1+4\cdot3 & 3\cdot2+4\cdot4 & 3\cdot1+4\cdot2 \end{array}\right]\\ & =\left[\begin{array}{ccc} 7 & 10 & 5\\ 15 & 22 & 11 \end{array}\right]=\underset{(2\times3)}{\mathbf{C}} \end{align*} The resulting matrix $$\mathbf{C}$$ has 2 rows and 3 columns. The $$(1,1)$$ element of $$\mathbf{C}$$ is the dot product of the first row of $$\mathbf{A}$$ with the first column of $$\mathbf{B}$$: $\left(\begin{array}{cc} 1 & 2\end{array}\right)\cdot\left(\begin{array}{c} 1\\ 3 \end{array}\right)=1\cdot1+2\cdot3=7.$ The $$(1,2)$$ element of $$\mathbf{C}$$ is the dot product of the first row of $$\mathbf{A}$$ with the second column of $$\mathbf{B}$$: $\left(\begin{array}{cc} 1 & 2\end{array}\right)\cdot\left(\begin{array}{c} 2\\ 4 \end{array}\right)=1\cdot2+2\cdot4=10.$ In general, the $$(i,j)$$ element of $$\mathbf{C}$$ is the dot product of the ith row of $$\mathbf{A}$$ with the jth column of $$\mathbf{B}$$. If $$\mathbf{A}$$ is $$n\times m$$ and $$\mathbf{B}$$ is $$m\times p$$ then $$\mathbf{C=A\cdot B}$$ is $$n\times p$$.

As another example, let, $\underset{(2\times2)}{\mathbf{A}}=\left[\begin{array}{cc} 1 & 2\\ 3 & 4 \end{array}\right]\textrm{ and}\underset{(2\times1)}{\mathbf{B}}=\left[\begin{array}{c} 2\\ 6 \end{array}\right].$ Then, \begin{align*} \underset{(2\times2)}{\mathbf{A}}\cdot\underset{(2\times1)}{\mathbf{B}} & =\left[\begin{array}{cc} 1 & 2\\ 3 & 4 \end{array}\right]\cdot\left[\begin{array}{c} 2\\ 6 \end{array}\right]\\ & =\left[\begin{array}{c} 1\cdot2+2\cdot6\\ 3\cdot2+4\cdot6 \end{array}\right]\\ & =\left[\begin{array}{c} 14\\ 30 \end{array}\right]. \end{align*} As a final example, let, $\mathbf{x}=\left[\begin{array}{c} 1\\ 2\\ 3 \end{array}\right],~\mathbf{y}=\left[\begin{array}{c} 4\\ 5\\ 6 \end{array}\right].$ Then $\mathbf{x}^{\prime}\mathbf{y}=\left[\begin{array}{ccc} 1 & 2 & 3\end{array}\right]\cdot\left[\begin{array}{c} 4\\ 5\\ 6 \end{array}\right]=1\cdot4+2\cdot5+3\cdot6=32$
Example 2.12 (Matrix multiplication in R)

In R, matrix multiplication is performed with the % * % operator. For example:

matA = matrix(1:4,2,2,byrow=TRUE)
matB = matrix(c(1,2,1,3,4,2),2,3,byrow=TRUE)
matA
##      [,1] [,2]
## [1,]    1    2
## [2,]    3    4
matB
##      [,1] [,2] [,3]
## [1,]    1    2    1
## [2,]    3    4    2
dim(matA)
## [1] 2 2
dim(matB)
## [1] 2 3
matC = matA%*%matB
matC
##      [,1] [,2] [,3]
## [1,]    7   10    5
## [2,]   15   22   11

The next example shows that matrix multiplication in R also works on numeric vectors:

matA = matrix(c(1,2,3,4), 2, 2, byrow=TRUE)
vecB = c(2,6)
matA%*%vecB
##      [,1]
## [1,]   14
## [2,]   30
vecX = c(1,2,3)
vecY = c(4,5,6)
t(vecX)%*%vecY
##      [,1]
## [1,]   32
crossprod(vecX, vecY)
##      [,1]
## [1,]   32

$$\blacksquare$$

#### 3.2.3.1 Miscellaneous properties

Matrix multiplication satisfies the associative property: Let $$\mathbf{A},\,\mathbf{B}\,$$ and $$\mathbf{C}$$ be conformable matrices. Then $\mathbf{A}(\mathbf{B}+\mathbf{C})=\mathbf{AB}+\mathbf{AC}.$ Also, the transpose of the product of two matrices is the product of the transposes in opposite order: $(\mathbf{AB})^{\prime}=\mathbf{B}^{\prime}\mathbf{A}^{\prime}.$

### 3.2.4 The identity matrix

The identity matrix plays a similar role as the number $$1.$$ Multiplying any number by $$1$$ gives back that number. In matrix algebra, pre-multiplying or post-multiplying a matrix $$\mathbf{A}$$ by a conformable identity matrix $$\mathbf{I}$$ gives back the matrix $$\mathbf{A}$$. To illustrate, let $\mathbf{I}_{2}=\left[\begin{array}{cc} 1 & 0\\ 0 & 1 \end{array}\right]$ denote the 2-dimensional identity matrix and let $\mathbf{A}=\left[\begin{array}{cc} a_{11} & a_{12}\\ a_{21} & a_{22} \end{array}\right]$ denote an arbitrary $$2\times2$$ matrix. Then, \begin{align*} \mathbf{I}_{2}\mathbf{\cdot A} & =\left[\begin{array}{cc} 1 & 0\\ 0 & 1 \end{array}\right]\cdot\left[\begin{array}{cc} a_{11} & a_{12}\\ a_{21} & a_{22} \end{array}\right]\\ & =\left[\begin{array}{cc} a_{11} & a_{12}\\ a_{21} & a_{22} \end{array}\right]=\mathbf{A}, \end{align*} and, \begin{align*} \mathbf{A\cdot I}_{2} & =\left[\begin{array}{cc} a_{11} & a_{12}\\ a_{21} & a_{22} \end{array}\right]\cdot\left[\begin{array}{cc} 1 & 0\\ 0 & 1 \end{array}\right]\\ & =\left[\begin{array}{cc} a_{11} & a_{12}\\ a_{21} & a_{22} \end{array}\right]=\mathbf{A}. \end{align*}

Example 3.4 (The identity matrix in R)

Use the diag() function to create an identity matrix:

matI = diag(2)
matI
##      [,1] [,2]
## [1,]    1    0
## [2,]    0    1
matA = matrix(c(1,2,3,4), 2, 2, byrow=TRUE)
matI%*%matA
##      [,1] [,2]
## [1,]    1    2
## [2,]    3    4
matA%*%matI
##      [,1] [,2]
## [1,]    1    2
## [2,]    3    4

$$\blacksquare$$

### 3.2.5 Diagonal, lower triangular and upper triangular matrices

Consider an $$n\times n$$ matrix $$\mathbf{A}$$ $\mathbf{A}=\left[\begin{array}{ccccc} d_{1} & u_{12} & u_{13} & \cdots & u_{1n}\\ l_{21} & d_{2} & u_{23} & \cdots & u_{2n}\\ l_{31} & l_{32} & d_{3} & \cdots & u_{3n}\\ \vdots & \vdots & \vdots & \ddots & \vdots\\ l_{n1} & l_{n2} & l_{n3} & \cdots & d_{n} \end{array}\right].$ The main diagonal consists of the $$n$$ elements $$\{d_{1},d_{2}\ldots d_{n}\}$$. The lower triangle consists of the $$n(n-1)/2$$ elements $$\{l_{21},l_{31},l_{32},\ldots,l_{n(n-1)}\}$$ below the main diagonal, and the upper triangle consists of the $$n(n-1)/2$$ elements $$\{u_{12},u_{13},u_{23},\ldots,u_{(n-1)n}\}$$ above the main diagonal.

An $$n-$$dimensional diagonal matrix $$\mathbf{D}$$ is an $$n\times n$$ square matrix with an $$n\times1$$ vector $$\mathbf{d}=(d_{1},\ldots,d_{n})^{\prime}$$ along the main diagonal and zeros elsewhere: $\mathbf{D}= \mathrm{diag}(\mathbf{d}) = \left[\begin{array}{ccccc} d_{1} & 0 & 0 & \cdots & 0\\ 0 & d_{2} & 0 & \cdots & 0\\ 0 & 0 & d_{3} & \cdots & 0\\ \vdots & \vdots & \vdots & \ddots & \vdots\\ 0 & 0 & 0 & \cdots & d_{n} \end{array}\right].$ An $$n\times n$$ lower triangular matrix $$\mathbf{L}$$ has all values above the main diagonal equal to zero: $\mathbf{L}=\left[\begin{array}{ccccc} d_{1} & 0 & 0 & \cdots & 0\\ l_{21} & d_{2} & 0 & \cdots & 0\\ l_{31} & l_{32} & d_{3} & \cdots & 0\\ \vdots & \vdots & \vdots & \ddots & \vdots\\ l_{n1} & l_{n2} & l_{n3} & \cdots & d_{n} \end{array}\right].$ An $$n\times n$$ upper triangular matrix $$\mathbf{U}$$ has all values below the main diagonal equal to zero: $\mathbf{U}=\left[\begin{array}{ccccc} d_{1} & u_{12} & u_{13} & \cdots & u_{1n}\\ 0 & d_{2} & u_{23} & \cdots & u_{2n}\\ 0 & 0 & d_{3} & \cdots & u_{3n}\\ \vdots & \vdots & \vdots & \ddots & \vdots\\ 0 & 0 & 0 & \cdots & d_{n} \end{array}\right].$

Example 2.14 (Creating a diagonal matrix in R)

Diagonal matrices can be created with the diag() function. To create a $$3\times3$$ diagonal matrix with $$\mbox{d}=(1,2,3)$$ along the main diagonal, use

matD = diag(1:3)
matD
##      [,1] [,2] [,3]
## [1,]    1    0    0
## [2,]    0    2    0
## [3,]    0    0    3

$$\blacksquare$$

Example 3.5 (Extracting lower and upper triangular elements of a square matrix)

The R functions lower.tri() and upper.tri() can be used to extract the lower and upper triangular parts of a square matrix:

matA = matrix(c(1,2,3,4,5,6,7,8,9), 3, 3)
matA
##      [,1] [,2] [,3]
## [1,]    1    4    7
## [2,]    2    5    8
## [3,]    3    6    9
lower.tri(matA)
##       [,1]  [,2]  [,3]
## [1,] FALSE FALSE FALSE
## [2,]  TRUE FALSE FALSE
## [3,]  TRUE  TRUE FALSE
upper.tri(matA)
##       [,1]  [,2]  [,3]
## [1,] FALSE  TRUE  TRUE
## [2,] FALSE FALSE  TRUE
## [3,] FALSE FALSE FALSE
matA[lower.tri(matA)]
## [1] 2 3 6
matA[upper.tri(matA)]
## [1] 4 7 8

$$\blacksquare$$