14.5 Autocorrelation Tests
Autocorrelation (also known as serial correlation) occurs when the error terms (\(\epsilon_t\)) in a regression model are correlated across observations, violating the assumption of independence in the classical linear regression model. This issue is particularly common in time-series data, where observations are ordered over time.
Consequences of Autocorrelation:
- OLS estimators remain unbiased but become inefficient, meaning they do not have the minimum variance among all linear unbiased estimators.
- Standard errors are biased, leading to unreliable hypothesis tests (e.g., \(t\)-tests and \(F\)-tests).
- Potential underestimation of standard errors, increasing the risk of Type I errors (false positives).
14.5.1 Durbin–Watson Test
The Durbin–Watson (DW) Test is the most widely used test for detecting first-order autocorrelation, where the current error term is correlated with the previous one:
\[ \epsilon_t = \rho \, \epsilon_{t-1} + u_t \]
Where:
\(\rho\) is the autocorrelation coefficient,
\(u_t\) is a white noise error term.
Hypotheses
- Null Hypothesis (\(H_0\)): No first-order autocorrelation (\(\rho = 0\)).
- Alternative Hypothesis (\(H_1\)): First-order autocorrelation exists (\(\rho \neq 0\)).
Durbin–Watson Test Statistic
The DW statistic is calculated as:
\[ DW = \frac{\sum_{t=2}^{n} (\hat{\epsilon}_t - \hat{\epsilon}_{t-1})^2}{\sum_{t=1}^{n} \hat{\epsilon}_t^2} \]
Where:
\(\hat{\epsilon}_t\) are the residuals from the regression,
\(n\) is the number of observations.
Decision Rule
- The DW statistic ranges from 0 to 4:
- DW ≈ 2: No autocorrelation.
- DW < 2: Positive autocorrelation.
- DW > 2: Negative autocorrelation.
For more precise interpretation:
Use Durbin–Watson critical values (\(d_L\) and \(d_U\)) to form decision boundaries.
If the test statistic falls into an inconclusive range, consider alternative tests like the Breusch–Godfrey test.
Advantages and Limitations
- Advantage: Simple to compute; specifically designed for detecting first-order autocorrelation.
- Limitation: Inconclusive in some cases; invalid when lagged dependent variables are included in the model.
14.5.2 Breusch–Godfrey Test
The Breusch–Godfrey (BG) Test is a more general approach that can detect higher-order autocorrelation (e.g., second-order, third-order) and is valid even when lagged dependent variables are included in the model (Breusch 1978; Godfrey 1978).
Hypotheses
- Null Hypothesis (\(H_0\)): No autocorrelation of any order (up to lag \(p\)).
- Alternative Hypothesis (\(H_1\)): Autocorrelation exists at some lag(s).
Procedure
Estimate the original regression model and obtain residuals \(\hat{\epsilon}_t\):
\[ y_t = \beta_0 + \beta_1 x_{1t} + \dots + \beta_k x_{kt} + \epsilon_t \]
Run an auxiliary regression by regressing the residuals on the original regressors plus \(p\) lagged residuals:
\[ \hat{\epsilon}_t = \alpha_0 + \alpha_1 x_{1t} + \dots + \alpha_k x_{kt} + \rho_1 \hat{\epsilon}_{t-1} + \dots + \rho_p \hat{\epsilon}_{t-p} + u_t \]
Calculate the test statistic:
\[ \text{BG} = n \cdot R^2_{\text{aux}} \]
Where:
- \(n\) is the sample size,
- \(R^2_{\text{aux}}\) is the \(R^2\) from the auxiliary regression.
Decision Rule:
Under \(H_0\), the BG statistic follows a chi-squared distribution with \(p\) degrees of freedom:
\[ \text{BG} \sim \chi^2_p \]
Reject \(H_0\) if the statistic exceeds the critical chi-squared value, indicating the presence of autocorrelation.
Advantages and Limitations
- Advantage: Can detect higher-order autocorrelation; valid with lagged dependent variables.
- Limitation: More computationally intensive than the Durbin–Watson test.
14.5.3 Ljung–Box Test (or Box–Pierce Test)
The Ljung–Box Test is a portmanteau test designed to detect autocorrelation at multiple lags simultaneously (Box and Pierce 1970; Ljung and Box 1978). It is commonly used in time-series analysis to check residual autocorrelation after model estimation (e.g., in ARIMA models).
Hypotheses
- Null Hypothesis (\(H_0\)): No autocorrelation up to lag \(h\).
- Alternative Hypothesis (\(H_1\)): Autocorrelation exists at one or more lags.
Ljung–Box Test Statistic
The Ljung–Box statistic is calculated as:
\[ Q = n(n + 2) \sum_{k=1}^{h} \frac{\hat{\rho}_k^2}{n - k} \]
Where:
- \(n\) = Sample size,
- \(h\) = Number of lags tested,
- \(\hat{\rho}_k\) = Sample autocorrelation at lag \(k\).
Decision Rule
Under \(H_0\), the \(Q\) statistic follows a chi-squared distribution with \(h\) degrees of freedom:
\[ Q \sim \chi^2_h \]
Reject \(H_0\) if \(Q\) exceeds the critical value, indicating significant autocorrelation.
Advantages and Limitations
- Advantage: Detects autocorrelation across multiple lags simultaneously.
- Limitation: Less powerful for detecting specific lag structures; sensitive to model misspecification.
14.5.4 Runs Test
The Runs Test is a non-parametric test that examines the randomness of residuals. It is based on the number of runs—sequences of consecutive residuals with the same sign.
Hypotheses
- Null Hypothesis (\(H_0\)): Residuals are randomly distributed (no autocorrelation).
- Alternative Hypothesis (\(H_1\)): Residuals exhibit non-random patterns (indicating autocorrelation).
Procedure
Classify residuals as positive or negative.
Count the number of runs: A run is a sequence of consecutive positive or negative residuals.
Calculate the expected number of runs under randomness:
\[ E(R) = \frac{2 n_+ n_-}{n} + 1 \]
Where:
- \(n_+\) = Number of positive residuals,
- \(n_-\) = Number of negative residuals,
- \(n = n_+ + n_-\).
Compute the test statistic (Z-score):
\[ Z = \frac{R - E(R)}{\sqrt{\text{Var}(R)}} \]
Where \(\text{Var}(R)\) is the variance of the number of runs under the null hypothesis.
Decision Rule
Under \(H_0\), the \(Z\)-statistic follows a standard normal distribution:
\[ Z \sim N(0, 1) \]
Reject \(H_0\) if \(|Z|\) exceeds the critical value from the standard normal distribution.
Advantages and Limitations
- Advantage: Non-parametric; does not assume normality or linearity.
- Limitation: Less powerful than parametric tests; primarily useful as a supplementary diagnostic.
14.5.5 Summary of Autocorrelation Tests
| Test | Type | Key Statistic | Detects | When to Use |
|---|---|---|---|---|
| Durbin–Watson | Parametric | \(DW\) | First-order autocorrelation | Simple linear models without lagged dependent variables |
| Breusch–Godfrey | Parametric (general) | \(\chi^2\) | Higher-order autocorrelation | Models with lagged dependent variables |
| Ljung–Box | Portmanteau (global test) | \(\chi^2\) | Autocorrelation at multiple lags | Time-series models (e.g., ARIMA) |
| Runs Test | Non-parametric | \(Z\)-statistic | Non-random patterns in residuals | Supplementary diagnostic for randomness |
Detecting autocorrelation is crucial for ensuring the efficiency and reliability of regression models, especially in time-series analysis. While the Durbin–Watson Test is suitable for detecting first-order autocorrelation, the Breusch–Godfrey Test and Ljung–Box Test offer more flexibility for higher-order and multi-lag dependencies. Non-parametric tests like the Runs Test serve as useful supplementary diagnostics.
# Install and load necessary libraries
# install.packages("lmtest") # For Durbin–Watson and Breusch–Godfrey Tests
# install.packages("tseries") # For Runs Test
# install.packages("forecast")# For Ljung–Box Test
library(lmtest)
library(tseries)
library(forecast)
# Simulated time-series dataset
set.seed(123)
n <- 100
time <- 1:n
x1 <- rnorm(n, mean = 50, sd = 10)
x2 <- rnorm(n, mean = 30, sd = 5)
# Introducing autocorrelation in errors
epsilon <- arima.sim(model = list(ar = 0.6), n = n)
# AR(1) process with ρ = 0.6
y <- 5 + 0.4 * x1 - 0.3 * x2 + epsilon
# Original regression model
model <- lm(y ~ x1 + x2)
# ----------------------------------------------------------------------
# 1. Durbin–Watson Test
# ----------------------------------------------------------------------
# Null Hypothesis: No first-order autocorrelation
dw_test <- dwtest(model)
print(dw_test)
#>
#> Durbin-Watson test
#>
#> data: model
#> DW = 0.77291, p-value = 3.323e-10
#> alternative hypothesis: true autocorrelation is greater than 0
# ----------------------------------------------------------------------
# 2. Breusch–Godfrey Test
# ----------------------------------------------------------------------
# Null Hypothesis: No autocorrelation up to lag 2
bg_test <- bgtest(model, order = 2) # Testing for autocorrelation up to lag 2
print(bg_test)
#>
#> Breusch-Godfrey test for serial correlation of order up to 2
#>
#> data: model
#> LM test = 40.314, df = 2, p-value = 1.762e-09
# ----------------------------------------------------------------------
# 3. Ljung–Box Test
# ----------------------------------------------------------------------
# Null Hypothesis: No autocorrelation up to lag 10
ljung_box_test <- Box.test(residuals(model), lag = 10, type = "Ljung-Box")
print(ljung_box_test)
#>
#> Box-Ljung test
#>
#> data: residuals(model)
#> X-squared = 50.123, df = 10, p-value = 2.534e-07
# ----------------------------------------------------------------------
# 4. Runs Test (Non-parametric)
# ----------------------------------------------------------------------
# Null Hypothesis: Residuals are randomly distributed
runs_test <- runs.test(as.factor(sign(residuals(model))))
print(runs_test)
#>
#> Runs Test
#>
#> data: as.factor(sign(residuals(model)))
#> Standard Normal = -4.2214, p-value = 2.428e-05
#> alternative hypothesis: two.sidedInterpretation of the Results
Durbin–Watson Test
Null Hypothesis (\(H_0\)): No first-order autocorrelation (\(\rho = 0\)).
Alternative Hypothesis (\(H_1\)): First-order autocorrelation exists (\(\rho \neq 0\)).
Decision Rule:
- Reject \(H_0\) if DW \(< 1.5\) (positive autocorrelation) or DW \(> 2.5\) (negative autocorrelation).
- Fail to reject \(H_0\) if DW \(\approx 2\), suggesting no significant autocorrelation.
Breusch–Godfrey Test
Null Hypothesis (\(H_0\)): No autocorrelation up to lag \(p\) (here, \(p = 2\)).
Alternative Hypothesis (\(H_1\)): Autocorrelation exists at one or more lags.
Decision Rule:
- Reject \(H_0\) if p-value \(< 0.05\), indicating significant autocorrelation.
- Fail to reject \(H_0\) if p-value \(\ge 0.05\), suggesting no evidence of autocorrelation.
Ljung–Box Test
Null Hypothesis (\(H_0\)): No autocorrelation up to lag \(h\) (here, \(h = 10\)).
Alternative Hypothesis (\(H_1\)): Autocorrelation exists at one or more lags.
Decision Rule:
- Reject \(H_0\) if p-value \(< 0.05\), indicating significant autocorrelation.
- Fail to reject \(H_0\) if p-value \(\ge 0.05\), suggesting no evidence of autocorrelation.
Runs Test (Non-parametric)
Null Hypothesis (\(H_0\)): Residuals are randomly distributed (no autocorrelation).
Alternative Hypothesis (\(H_1\)): Residuals exhibit non-random patterns (indicating autocorrelation).
Decision Rule:
- Reject \(H_0\) if p-value \(< 0.05\), indicating non-randomness and potential autocorrelation.
- Fail to reject \(H_0\) if p-value \(\ge 0.05\), suggesting randomness in residuals.