Introduction to the course
Before each lecture
What to expect in each lecture
What to do after each lecture
Moodle
Connections with other courses
Mathematics skills
Glossary of terms
Summary of Course
Lecture 1
Examples
Air Temperature
Power
Alcohol consumed and blood alcohol content
Assessing the relationship between two variables
Crime
Potatoes
Summary points
Additional Reading
Lecture 2
Defining a statistical model
Examples
Air Temperature
Power
Alcohol consumed and blood alcohol content
Crime
Potatoes
What is a linear model?
Functions that are examples of linear models
Functions that are not examples of linear models
Example
X-ray
Simple linear Regression And Multiple Linear Regression
Notation
Summary points
Additional Reading
Lecture 3
The Method of Least Squares
Why least squares?
Least squares estimate for simple linear regression through the origin
Least squares estimators
Residual sum of squares (RSS)
Minimisation of
\(S(\beta)\)
Steps for calculus solution to least square estimate
Common notation and sum of square functions
Least squares estimates for a simple linear regression
Protein in pregnancy
Summary points
Additional Reading
Lecture 4
The Simple Linear Regression
Examples
X-ray
Potatoes
Vector-matrix form of a linear model
Construction of
\(S(\boldsymbol{\beta})\)
Least Squares Estimates For The Linear Regression
Residual Sum Of Squares
Use of the general least square estimate
Summary points
Additional Reading
Lecture 5
Correlation
Calculation of the correlation coefficient
Interpretation of
\(r\)
Typical scatterplots
Example
Power
Inference about
\(\rho\)
For information only
Example
Power
Effect of laser treatment on pain felt during dental procedure
Plot the data
Summary points
Additional Reading
Lecture 6
A summary of the fitted model
The Simple Linear Regression
Sums of Squares
Coefficient of Determination
\(R^2\)
Nested Models
The Multiple Linear Regression
\(R^2\)
(adj) as a measure of model fit
Examples
Protein in Pregnancy
Giving in the Church of England
Interpretation of
\(R^2\)
and
\(R^2 \mbox(adj)\)
Analysis Of Variance (ANOVA)
Analysis Of Variance (ANOVA) Table Construction
ANOVA For The Simple Linear Regression
ANOVA For The Multiple Linear Regression
Examples
Protein in Pregnancy
Summary points
Additional reading
Lecture 7
Types of Data
Continuous variables
Binary variables
Binary variables in a regression setting
Specifying a regression model with categorical variables
Vector Matrix Notation
Vector matrix notation with binary variables
Categorical Variables
Categorical variables in a regression setting
Specifying a regression model with categorical variables
Example
Discrete variables
Summary points
Additional Reading
Lecture 8
Crime
Interpreting with categorical variables
Height, weight and gender
Month
Interpreting multiple regression
Height, weight and gender
Alternative notation
Interactions
Fitting a linear regression in
Height, weight and gender example in R
Different lines
Parallel lines
Single line
Summary points
Additional Reading
Lecture 9
R code
Example
Protein in Pregnancy
Residuals
Defining Residuals
Estimating the error variance
Regression Model Assumptions
The deterministic part of the model captures all the non-random structure in the data
The scale of the variability of the errors is constant at all values of the explanatory variables
The errors are independent
The errors are normally distributed
The values of the explanatory variables are recorded without error
Checking assumptions
Q-Q plot
Protein in Pregnancy
Interpreting Residual Plots
Summary ponts
Additional Reading
Lecture 10
R code
Departures from assumptions
Non-random structure
Independence
Non-constant variance
Normality
Examples
Tree volume
Transformations
Mass and speed of quadrupedal rodents
Cook’s distance and Leverage
Summary points
Additional Reading
Lecture 11
R code
Linear Combinations of Parameters
The least-squares estimate of linear functions of the parameters in a multiple linear model
Application of linear transformation of parameters for two parameter case
Example
Application of linear transformation of parameters for three parameter case
Inferences from regression equations
Pivotal function for a linear function of the parameters
Hypothesis Testing
Example
Protein in Pregnancy
Analysing the ANOVA table
Example
Summary points
Additional Reading
Lecture 12
R code
Interval estimate for
\(\mathbf{b}^T\boldsymbol{\beta}\)
Prediction interval (PI) for Y given x
Simple linear regression
Examples
Protein in pregnancy
Confidence Interval
Prediction Interval
Trees
Confidence interval for the difference in two population means
Autoanalyser data
Model diagnostics
Confidence Interval
Prediction interval
Summary points
Additional reading
Lecture 13
R code
Regression models with factors and interactions
Different lines
Model 1: Different slope and intercept terms
95% Confidence interval for slope parameters for two regression lines
Interpreting the the confidence interval
Parallel Lines
95% Confidence interval for parallel lines model
Interperetation of Confidence Interval
ANalysis of COVAriance
Summary points
Additional Reading
Lecture 14
R code
Trout
95% Confidence interval for parallel lines model
Paralle lines model in
R
Respiratory Distress Syndrome
Weight changes in Herring Gulls
Summary point
Lecture 15
R code
Variable selection with two continuous explanatory variables
Tree volume
Regression Analysis: log volume versus log diameter
Regression Analysis: log volume versus log height
Giving in the Church of England
Regression Analysis: Annual giving versus Employment rate and % of Attendance
Regression Analysis: Annual giving versus Employment
Regression Analysis: Annual giving versus % of Attendance
Confidence intervals
Annual giving versus Employment rate and % of Attendance
Annual giving versus Employment rate
Additional Reading
Lecture 16
R code
Variable selection with several explanatory variables
Example
Model selection for Motor Trend Car Road Tests data
Multicollinearity
Methods for variable selection
Selection criterion
Search strategy
All subset regression
Example
Model selection for Motor Trend Car Road Tests data
Number of models
Summary pooint
Additional Reading
Lecture 17
R code
Search strategy
Model selection for Motor Trend Car Road Tests data
Stepwise regression
Backward selection
Forward selection
Forward-Backward selection
Discussion
Summary point
Additional Reading
Lecture 18
R code
Example
Backward selection with AIC
Backward selection p-values
Forward-Backward selection with BIC
Lecture 19
Maximum Likelihood
Re-framing the linear model
Simple linear regression through the origin
Likelihood and Log-likelihood function
Simple linear regression with a slope and intercept term
Likelihood and Log-likelihood function
Maximum likelihood or least squares
Regression Models (Level M)
Additional Reading
You may find the following additional reading helpful.
Chapter 1 and section 2.1 in
An Introduction to Statistical Learning
Chapter 1 in
Linear Models with R