3 Day 3 (June 11)

3.1 Announcements

• If office hours times don’t work for you let me know

• Recommended reading

  • Chapters 1 and 2 (pgs 1 - 27) in Linear Models with R

3.2 Matrix algebra

• Column vectors
  • $\mathbf{y}\equiv(y_{1},y_{2},\ldots,y_{n})^{'}$
  • $\mathbf{x}\equiv(x_{1},x_{2},\ldots,x_{n})^{'}$
  • $\boldsymbol{\beta}\equiv(\beta_{1},\beta_{2},\ldots,\beta_{p})^{'}$
  • $\boldsymbol{1}\equiv(1,1,\ldots,1)^{'}$
  • In R

  y <- matrix(c(1,2,3),nrow=3,ncol=1)
  y

  ##      [,1]
  ## [1,]    1
  ## [2,]    2
  ## [3,]    3

• Matrices
  • $\mathbf{X}\equiv(\mathbf{x}_{1},\mathbf{x}_{2},\ldots,\mathbf{x}_{p})$
  • In R

  X <- matrix(c(1,2,3,4,5,6),nrow=3,ncol=2,byrow=FALSE)
  X

  ##      [,1] [,2]
  ## [1,]    1    4
  ## [2,]    2    5
  ## [3,]    3    6

• Vector multiplication
  • $\mathbf{y}^{'}\mathbf{y}$
  • $\mathbf{1}^{'}\mathbf{1}$
  • $\mathbf{1}\mathbf{1}^{'}$
  • In R

  t(y)%*%y    

  ##      [,1]
  ## [1,]   14

• Matrix by vector multiplication
  • $\mathbf{X}^{'}\mathbf{y}$
  • In R

  t(X)%*%y

  ##      [,1]
  ## [1,]   14
  ## [2,]   32

• Matrix by matrix multiplication
  • $\mathbf{X}^{'}\mathbf{X}$
  • In R

  t(X)%*%X

  ##      [,1] [,2]
  ## [1,]   14   32
  ## [2,]   32   77

• Matrix inversion
  • $(\mathbf{X}^{'}\mathbf{X})^{-1}$
  • In R

  solve(t(X)%*%X)

  ##            [,1]       [,2]
  ## [1,]  1.4259259 -0.5925926
  ## [2,] -0.5925926  0.2592593

• Determinant of a matrix
  • $|\mathbf{I}|$
    
  • In R

  I <- diag(1,3)
  I

  ##      [,1] [,2] [,3]
  ## [1,]    1    0    0
  ## [2,]    0    1    0
  ## [3,]    0    0    1

  det(I)

  ## [1] 1

• Quadratic form
  • $\mathbf{y}^{'}\mathbf{S}\mathbf{y}$
• Derivative of a quadratic form (Note $\mathbf{S}$ is a symmetric matrix; e.g., $\mathbf{X}^{'}\mathbf{X}$)
  • $\frac{\partial}{\partial\mathbf{y}}\mathbf{y^{'}\mathbf{S}\mathbf{y}}=2\mathbf{S}\mathbf{y}$
• Other useful derivatives
  • $\frac{\partial}{\partial\mathbf{y}}\mathbf{\mathbf{x^{'}}\mathbf{y}}=\mathbf{x}$
  • $\frac{\partial}{\partial\mathbf{y}}\mathbf{\mathbf{X^{'}}\mathbf{y}}=\mathbf{X}$