4 Day 4 (June 6)

4.1 Announcements

• Tutoring for program R

  • Dickens Hall room 108
  • 12:30 - 1:30 Monday - Friday

• Recommended reading

  • Chapters 1 and 2 (pgs 1 - 28) in Linear Models with R
  • Chapter 2 in Applied Regression and ANOVA Using SAS

• Final project is posted

• Assignment 2 is posted a due Wednesday June 12

• Special in-class event on Friday!

4.2 Introduction to linear models

• What is a model?

• What is a linear model?

  • Most widely used model in science, engineering, and statistics

  • Vector form: \(\mathbf{y}=\beta_{0}+\beta_{1}\mathbf{x}_{1}+\beta_{2}\mathbf{x}_{2}+\ldots+\beta_{p}\mathbf{x}_{p}+\boldsymbol{\varepsilon}\)

  • Matrix form: \(\mathbf{y}=\mathbf{X}\boldsymbol{\beta}+\boldsymbol{\varepsilon}\)

  • Which part of the model is the mathematical model

  • Which part of the model makes the linear model a “statistical” model

  • Visual

• Which of the four below are a linear model \[\mathbf{y}=\beta_{0}+\beta_{1}\mathbf{x}_{1}+\beta_{2}\mathbf{x}^{2}_{1}+\boldsymbol{\varepsilon}\] \[\mathbf{y}=\beta_{0}+\beta_{1}\mathbf{x}_{1}+\beta_{2}\text{log(}\mathbf{x}_{1}\text{)}+\boldsymbol{\varepsilon}\] \[\mathbf{y}=\beta_{0}+\beta_{1}e^{\beta_{2}\mathbf{x}_{1}}+\boldsymbol{\varepsilon}\] \[\mathbf{y}=\beta_{0}+\beta_{1}\mathbf{x}_{1}+\text{log(}\beta_{2}\text{)}\mathbf{x}_{1}+\boldsymbol{\varepsilon}\]

• Why study the linear model?

  • Building block for more complex models (e.g., GLMs, mixed models, machine learning, etc)
  • We know the most about it

4.3 Estimation

• Three options to estimate \(\boldsymbol{\beta}\)
  • Minimize a loss function
  • Maximize a likelihood function
  • Find the posterior distribution
  • Each option requires different assumptions

4.4 Loss function approach

• Define a measure of discrepancy between the data and the mathematical model
  • Find the values of \(\boldsymbol{\beta}\) that make \(\mathbf{X}\boldsymbol{\beta}\) “closest” to \(\mathbf{y}\)
  • Visual
• Classic example \[\underset{\boldsymbol{\beta}}{\operatorname{argmin}}\sum_{i=1}^{n}(y_i-\mathbf{x}_{i}^{\prime}\boldsymbol{\beta})^2\] or in matrix form \[\underset{\boldsymbol{\beta}}{\operatorname{argmin}}(\mathbf{y} - \mathbf{X}\boldsymbol{\beta})^{\prime}(\mathbf{y} - \mathbf{X}\boldsymbol{\beta})\] which results in \[\hat{\boldsymbol{\beta}}=(\mathbf{X}^{\prime}\mathbf{X})^{-1}\mathbf{X}^{\prime}\mathbf{y}\]
• Three ways to do it in program R
  • Using scalar calculus and algebra (kind of)

  y <- c(0.16,2.82,2.24)
  x <- c(1,2,3)
  
  y.bar <- mean(y)
  x.bar <- mean(x)
  
  # Estimate the slope parameter
  beta1.hat <- sum((x-x.bar)*(y-y.bar))/sum((x-x.bar)^2)
  beta1.hat

  ## [1] 1.04

  # Estimate the intercept parameter
  beta0.hat <- y.bar - sum((x-x.bar)*(y-y.bar))/sum((x-x.bar)^2)*x.bar
  beta0.hat

  ## [1] -0.34