3 Day 3 (June 5)
3.1 Announcements
If office hours times don’t work for you let me know
Recommended reading
- Chapters 1 and 2 (pgs 1 - 28) in Linear Models with R
- Chapter 2 in Applied Regression and ANOVA Using SAS
3.2 Matrix algebra
- Column vectors
- y≡(y1,y2,…,yn)′
- x≡(x1,x2,…,xn)′
- β≡(β1,β2,…,βp)′
- 1≡(1,1,…,1)′
- In R
## [,1] ## [1,] 1 ## [2,] 2 ## [3,] 3
- Matrices
- X≡(x1,x2,…,xp)
- In R
## [,1] [,2] ## [1,] 1 4 ## [2,] 2 5 ## [3,] 3 6
- Vector multiplication
- y′y
- 1′1
- 11′
- In R
## [,1] ## [1,] 14
- Matrix by vector multiplication
- X′y
- In R
## [,1] ## [1,] 14 ## [2,] 32
- Matrix by matrix multiplication
- X′X
- In R
## [,1] [,2] ## [1,] 14 32 ## [2,] 32 77
- Matrix inversion
- (X′X)−1
- In R
## [,1] [,2] ## [1,] 1.4259259 -0.5925926 ## [2,] -0.5925926 0.2592593
- Determinant of a matrix
- |I|
- In R
## [,1] [,2] [,3] ## [1,] 1 0 0 ## [2,] 0 1 0 ## [3,] 0 0 1
## [1] 1
- |I|
- Quadratic form
- y′Sy
- Derivative of a quadratic form (Note S is a symmetric matrix; e.g., X′X)
- ∂∂yy′Sy=2Sy
- Other useful derivatives
- ∂∂yx′y=x
- ∂∂yX′y=X
3.3 Introduction to linear models
What is a model?
What is a linear model?
Most widely used model in science, engineering, and statistics
Vector form: y=β0+β1x1+β2x2+…+βpxp+ε
Matrix form: y=Xβ+ε
Which part of the model is the mathematical model
Which part of the model makes the linear model a “statistical” model
Visual
Which of the four below are a linear model y=β0+β1x1+β2x21+ε y=β0+β1x1+β2log(x1)+ε y=β0+β1eβ2x1+ε y=β0+β1x1+log(β2)x1+ε
Why study the linear model?
- Building block for more complex models (e.g., GLMs, mixed models, machine learning, etc)
- We know the most about it