3 Day 3 (June 5)

3.1 Announcements

  • If office hours times don’t work for you let me know

  • Recommended reading

    • Chapters 1 and 2 (pgs 1 - 28) in Linear Models with R
    • Chapter 2 in Applied Regression and ANOVA Using SAS

3.2 Matrix algebra

  • Column vectors
    • y(y1,y2,,yn)
    • x(x1,x2,,xn)
    • β(β1,β2,,βp)
    • 1(1,1,,1)
    • In R
    y <- matrix(c(1,2,3),nrow=3,ncol=1)
    y
    ##      [,1]
    ## [1,]    1
    ## [2,]    2
    ## [3,]    3
  • Matrices
    • X(x1,x2,,xp)
    • In R
    X <- matrix(c(1,2,3,4,5,6),nrow=3,ncol=2,byrow=FALSE)
    X
    ##      [,1] [,2]
    ## [1,]    1    4
    ## [2,]    2    5
    ## [3,]    3    6
  • Vector multiplication
    • yy
    • 11
    • 11
    • In R
    t(y)%*%y    
    ##      [,1]
    ## [1,]   14
  • Matrix by vector multiplication
    • Xy
    • In R
    t(X)%*%y
    ##      [,1]
    ## [1,]   14
    ## [2,]   32
  • Matrix by matrix multiplication
    • XX
    • In R
    t(X)%*%X
    ##      [,1] [,2]
    ## [1,]   14   32
    ## [2,]   32   77
  • Matrix inversion
    • (XX)1
    • In R
    solve(t(X)%*%X)
    ##            [,1]       [,2]
    ## [1,]  1.4259259 -0.5925926
    ## [2,] -0.5925926  0.2592593
  • Determinant of a matrix
    • |I|
    • In R
    I <- diag(1,3)
    I
    ##      [,1] [,2] [,3]
    ## [1,]    1    0    0
    ## [2,]    0    1    0
    ## [3,]    0    0    1
    det(I)
    ## [1] 1
  • Quadratic form
    • ySy
  • Derivative of a quadratic form (Note S is a symmetric matrix; e.g., XX)
    • yySy=2Sy
  • Other useful derivatives
    • yxy=x
    • yXy=X

3.3 Introduction to linear models

  • What is a model?

  • What is a linear model?

    • Most widely used model in science, engineering, and statistics

    • Vector form: y=β0+β1x1+β2x2++βpxp+ε

    • Matrix form: y=Xβ+ε

    • Which part of the model is the mathematical model

    • Which part of the model makes the linear model a “statistical” model

    • Visual

  • Which of the four below are a linear model y=β0+β1x1+β2x21+ε y=β0+β1x1+β2log(x1)+ε y=β0+β1eβ2x1+ε y=β0+β1x1+log(β2)x1+ε

  • Why study the linear model?

    • Building block for more complex models (e.g., GLMs, mixed models, machine learning, etc)
    • We know the most about it