5.9 Robust Regression

• To address the problem of influential cases.
• Can be used when a known functional form is to be fitted, and when the errors are not normal due to a few outlying cases.

5.9.1 Least Absolute Residuals (LAR) Regression

also known as minimum \(L_1\)-norm regression.

\[ L_1 = \sum_{i=1}^{n}|Y_i - (\beta_0 + \beta_1 X_{i1} + .. + \beta_{p-1}X_{i,p-1}) \]

which is not sensitive to outliers and inadequacies of the model specification.

5.9.2 Least Median of Squares (LMS) Regression

\[ median\{[Y_i - (\beta_0 - \beta_1X_{i1} + ... + \beta_{p-1}X_{i,p-1})]^2 \} \]

5.9.3 Iteratively Reweighted Least Squares (IRLS) Robust Regression

• uses Weighted Least Squares to lessen the influence of outliers.
• the weights \(w_i\) are inversely proportional to how far an outlying case is (e.g., based on the residual)
• the weights are revised iteratively until a robust fit

Process:

Step 1: Choose a weight function for weighting the cases.
Step 2: obtain starting weights for all cases.
Step 3: Use the starting weights in WLS and obtain the residuals from the fitted regression function.
Step 4: use the residuals in Step 3 to obtain revised weights.
Step 5: continue until convergence.

Note:

• If you don’t know the form of the regression function, consider using nonparametric regression (e.g., locally weighted regression, regression trees, projection pursuit, neural networks, smoothing splines, loess, wavelets).
• could use to detect outliers or confirm OLS.