14.5 Autocorrelation Tests

Autocorrelation (also known as serial correlation) occurs when the error terms (\(\epsilon_t\)) in a regression model are correlated across observations, violating the assumption of independence in the classical linear regression model. This issue is particularly common in time-series data, where observations are ordered over time.

Consequences of Autocorrelation:

  • OLS estimators remain unbiased but become inefficient, meaning they do not have the minimum variance among all linear unbiased estimators.
  • Standard errors are biased, leading to unreliable hypothesis tests (e.g., \(t\)-tests and \(F\)-tests).
  • Potential underestimation of standard errors, increasing the risk of Type I errors (false positives).

14.5.1 Durbin–Watson Test

The Durbin–Watson (DW) Test is the most widely used test for detecting first-order autocorrelation, where the current error term is correlated with the previous one:

\[ \epsilon_t = \rho \, \epsilon_{t-1} + u_t \]

Where:

  • \(\rho\) is the autocorrelation coefficient,

  • \(u_t\) is a white noise error term.


Hypotheses

  • Null Hypothesis (\(H_0\)): No first-order autocorrelation (\(\rho = 0\)).
  • Alternative Hypothesis (\(H_1\)): First-order autocorrelation exists (\(\rho \neq 0\)).

Durbin–Watson Test Statistic

The DW statistic is calculated as:

\[ DW = \frac{\sum_{t=2}^{n} (\hat{\epsilon}_t - \hat{\epsilon}_{t-1})^2}{\sum_{t=1}^{n} \hat{\epsilon}_t^2} \]

Where:

  • \(\hat{\epsilon}_t\) are the residuals from the regression,

  • \(n\) is the number of observations.


Decision Rule

  • The DW statistic ranges from 0 to 4:
    • DW ≈ 2: No autocorrelation.
    • DW < 2: Positive autocorrelation.
    • DW > 2: Negative autocorrelation.

For more precise interpretation:

  • Use Durbin–Watson critical values (\(d_L\) and \(d_U\)) to form decision boundaries.

  • If the test statistic falls into an inconclusive range, consider alternative tests like the Breusch–Godfrey test.


Advantages and Limitations

  • Advantage: Simple to compute; specifically designed for detecting first-order autocorrelation.
  • Limitation: Inconclusive in some cases; invalid when lagged dependent variables are included in the model.

14.5.2 Breusch–Godfrey Test

The Breusch–Godfrey (BG) Test is a more general approach that can detect higher-order autocorrelation (e.g., second-order, third-order) and is valid even when lagged dependent variables are included in the model (Breusch 1978; Godfrey 1978).


Hypotheses

  • Null Hypothesis (\(H_0\)): No autocorrelation of any order (up to lag \(p\)).
  • Alternative Hypothesis (\(H_1\)): Autocorrelation exists at some lag(s).

Procedure

  1. Estimate the original regression model and obtain residuals \(\hat{\epsilon}_t\):

    \[ y_t = \beta_0 + \beta_1 x_{1t} + \dots + \beta_k x_{kt} + \epsilon_t \]

  2. Run an auxiliary regression by regressing the residuals on the original regressors plus \(p\) lagged residuals:

    \[ \hat{\epsilon}_t = \alpha_0 + \alpha_1 x_{1t} + \dots + \alpha_k x_{kt} + \rho_1 \hat{\epsilon}_{t-1} + \dots + \rho_p \hat{\epsilon}_{t-p} + u_t \]

  3. Calculate the test statistic:

    \[ \text{BG} = n \cdot R^2_{\text{aux}} \]

    Where:

    • \(n\) is the sample size,
    • \(R^2_{\text{aux}}\) is the \(R^2\) from the auxiliary regression.

Decision Rule:

  • Under \(H_0\), the BG statistic follows a chi-squared distribution with \(p\) degrees of freedom:

    \[ \text{BG} \sim \chi^2_p \]

  • Reject \(H_0\) if the statistic exceeds the critical chi-squared value, indicating the presence of autocorrelation.


Advantages and Limitations

  • Advantage: Can detect higher-order autocorrelation; valid with lagged dependent variables.
  • Limitation: More computationally intensive than the Durbin–Watson test.

14.5.3 Ljung–Box Test (or Box–Pierce Test)

The Ljung–Box Test is a portmanteau test designed to detect autocorrelation at multiple lags simultaneously (Box and Pierce 1970; Ljung and Box 1978). It is commonly used in time-series analysis to check residual autocorrelation after model estimation (e.g., in ARIMA models).


Hypotheses

  • Null Hypothesis (\(H_0\)): No autocorrelation up to lag \(h\).
  • Alternative Hypothesis (\(H_1\)): Autocorrelation exists at one or more lags.

Ljung–Box Test Statistic

The Ljung–Box statistic is calculated as:

\[ Q = n(n + 2) \sum_{k=1}^{h} \frac{\hat{\rho}_k^2}{n - k} \]

Where:

  • \(n\) = Sample size,
  • \(h\) = Number of lags tested,
  • \(\hat{\rho}_k\) = Sample autocorrelation at lag \(k\).

Decision Rule

  • Under \(H_0\), the \(Q\) statistic follows a chi-squared distribution with \(h\) degrees of freedom:

    \[ Q \sim \chi^2_h \]

  • Reject \(H_0\) if \(Q\) exceeds the critical value, indicating significant autocorrelation.


Advantages and Limitations

  • Advantage: Detects autocorrelation across multiple lags simultaneously.
  • Limitation: Less powerful for detecting specific lag structures; sensitive to model misspecification.

14.5.4 Runs Test

The Runs Test is a non-parametric test that examines the randomness of residuals. It is based on the number of runs—sequences of consecutive residuals with the same sign.


Hypotheses

  • Null Hypothesis (\(H_0\)): Residuals are randomly distributed (no autocorrelation).
  • Alternative Hypothesis (\(H_1\)): Residuals exhibit non-random patterns (indicating autocorrelation).

Procedure

  1. Classify residuals as positive or negative.

  2. Count the number of runs: A run is a sequence of consecutive positive or negative residuals.

  3. Calculate the expected number of runs under randomness:

    \[ E(R) = \frac{2 n_+ n_-}{n} + 1 \]

    Where:

    • \(n_+\) = Number of positive residuals,
    • \(n_-\) = Number of negative residuals,
    • \(n = n_+ + n_-\).
  4. Compute the test statistic (Z-score):

    \[ Z = \frac{R - E(R)}{\sqrt{\text{Var}(R)}} \]

    Where \(\text{Var}(R)\) is the variance of the number of runs under the null hypothesis.


Decision Rule

  • Under \(H_0\), the \(Z\)-statistic follows a standard normal distribution:

    \[ Z \sim N(0, 1) \]

  • Reject \(H_0\) if \(|Z|\) exceeds the critical value from the standard normal distribution.


Advantages and Limitations

  • Advantage: Non-parametric; does not assume normality or linearity.
  • Limitation: Less powerful than parametric tests; primarily useful as a supplementary diagnostic.

14.5.5 Summary of Autocorrelation Tests

Test Type Key Statistic Detects When to Use
Durbin–Watson Parametric \(DW\) First-order autocorrelation Simple linear models without lagged dependent variables
Breusch–Godfrey Parametric (general) \(\chi^2\) Higher-order autocorrelation Models with lagged dependent variables
Ljung–Box Portmanteau (global test) \(\chi^2\) Autocorrelation at multiple lags Time-series models (e.g., ARIMA)
Runs Test Non-parametric \(Z\)-statistic Non-random patterns in residuals Supplementary diagnostic for randomness

Detecting autocorrelation is crucial for ensuring the efficiency and reliability of regression models, especially in time-series analysis. While the Durbin–Watson Test is suitable for detecting first-order autocorrelation, the Breusch–Godfrey Test and Ljung–Box Test offer more flexibility for higher-order and multi-lag dependencies. Non-parametric tests like the Runs Test serve as useful supplementary diagnostics.

# Install and load necessary libraries
# install.packages("lmtest")  # For Durbin–Watson and Breusch–Godfrey Tests
# install.packages("tseries") # For Runs Test
# install.packages("forecast")# For Ljung–Box Test

library(lmtest)
library(tseries)
library(forecast)

# Simulated time-series dataset
set.seed(123)
n <- 100
time <- 1:n
x1 <- rnorm(n, mean = 50, sd = 10)
x2 <- rnorm(n, mean = 30, sd = 5)

# Introducing autocorrelation in errors
epsilon <- arima.sim(model = list(ar = 0.6), n = n) 
# AR(1) process with ρ = 0.6
y <- 5 + 0.4 * x1 - 0.3 * x2 + epsilon

# Original regression model
model <- lm(y ~ x1 + x2)

# ----------------------------------------------------------------------
# 1. Durbin–Watson Test
# ----------------------------------------------------------------------
# Null Hypothesis: No first-order autocorrelation
dw_test <- dwtest(model)
print(dw_test)
#> 
#>  Durbin-Watson test
#> 
#> data:  model
#> DW = 0.77291, p-value = 3.323e-10
#> alternative hypothesis: true autocorrelation is greater than 0

# ----------------------------------------------------------------------
# 2. Breusch–Godfrey Test
# ----------------------------------------------------------------------
# Null Hypothesis: No autocorrelation up to lag 2
bg_test <- bgtest(model, order = 2)  # Testing for autocorrelation up to lag 2
print(bg_test)
#> 
#>  Breusch-Godfrey test for serial correlation of order up to 2
#> 
#> data:  model
#> LM test = 40.314, df = 2, p-value = 1.762e-09

# ----------------------------------------------------------------------
# 3. Ljung–Box Test
# ----------------------------------------------------------------------
# Null Hypothesis: No autocorrelation up to lag 10
ljung_box_test <- Box.test(residuals(model), lag = 10, type = "Ljung-Box")
print(ljung_box_test)
#> 
#>  Box-Ljung test
#> 
#> data:  residuals(model)
#> X-squared = 50.123, df = 10, p-value = 2.534e-07

# ----------------------------------------------------------------------
# 4. Runs Test (Non-parametric)
# ----------------------------------------------------------------------
# Null Hypothesis: Residuals are randomly distributed
runs_test <- runs.test(as.factor(sign(residuals(model))))
print(runs_test)
#> 
#>  Runs Test
#> 
#> data:  as.factor(sign(residuals(model)))
#> Standard Normal = -4.2214, p-value = 2.428e-05
#> alternative hypothesis: two.sided

Interpretation of the Results

  1. Durbin–Watson Test

    • Null Hypothesis (\(H_0\)): No first-order autocorrelation (\(\rho = 0\)).

    • Alternative Hypothesis (\(H_1\)): First-order autocorrelation exists (\(\rho \neq 0\)).

    • Decision Rule:

      • Reject \(H_0\) if DW \(< 1.5\) (positive autocorrelation) or DW \(> 2.5\) (negative autocorrelation).
      • Fail to reject \(H_0\) if DW \(\approx 2\), suggesting no significant autocorrelation.
  2. Breusch–Godfrey Test

    • Null Hypothesis (\(H_0\)): No autocorrelation up to lag \(p\) (here, \(p = 2\)).

    • Alternative Hypothesis (\(H_1\)): Autocorrelation exists at one or more lags.

    • Decision Rule:

      • Reject \(H_0\) if p-value \(< 0.05\), indicating significant autocorrelation.
      • Fail to reject \(H_0\) if p-value \(\ge 0.05\), suggesting no evidence of autocorrelation.
  3. Ljung–Box Test

    • Null Hypothesis (\(H_0\)): No autocorrelation up to lag \(h\) (here, \(h = 10\)).

    • Alternative Hypothesis (\(H_1\)): Autocorrelation exists at one or more lags.

    • Decision Rule:

      • Reject \(H_0\) if p-value \(< 0.05\), indicating significant autocorrelation.
      • Fail to reject \(H_0\) if p-value \(\ge 0.05\), suggesting no evidence of autocorrelation.
  4. Runs Test (Non-parametric)

    • Null Hypothesis (\(H_0\)): Residuals are randomly distributed (no autocorrelation).

    • Alternative Hypothesis (\(H_1\)): Residuals exhibit non-random patterns (indicating autocorrelation).

    • Decision Rule:

      • Reject \(H_0\) if p-value \(< 0.05\), indicating non-randomness and potential autocorrelation.
      • Fail to reject \(H_0\) if p-value \(\ge 0.05\), suggesting randomness in residuals.

References

Box, George EP, and David A Pierce. 1970. “Distribution of Residual Autocorrelations in Autoregressive-Integrated Moving Average Time Series Models.” Journal of the American Statistical Association 65 (332): 1509–26.
Breusch, Trevor S. 1978. “Testing for Autocorrelation in Dynamic Linear Models.” Australian Economic Papers 17 (31): 334–55.
Godfrey, Leslie G. 1978. “Testing Against General Autoregressive and Moving Average Error Models When the Regressors Include Lagged Dependent Variables.” Econometrica: Journal of the Econometric Society, 1293–1301.
Ljung, Greta M, and George EP Box. 1978. “On a Measure of Lack of Fit in Time Series Models.” Biometrika 65 (2): 297–303.