Chapter 5 Multiple Linear Models

Part II presents some statistical methods relating to Travel-Urban Form models. Chapter 5 to 10 introduce the main methods could be applied on VMT-urban form models because the response are continue variables. The models of mode choice are placed in Chapter of Generalized linear models. The last chapter Meta-Analysis introduces basic ideas and approaches of meta-analysis and dealing with publication bias.

These contents focus on some analysis which often be neglected or omitted in travel-urban form models and may give some inspiration for improving current studies. Hence, these are mostly from several textbooks (Casella and Berger 2002; Montgomery, Peck, and Vining 2021; Ravishanker and Dey 2020) and lecture notes by Dr. Robert Fountain, Dr. Nadeeshani Jayasena, and Dr. Jong Sung Kim. It will not involve the front edge of techniques in recent published papers in statistics.

5.1 Assumptions

5.1.1 Additive and linearity

For linear models, the most important assumptions are the additive and linear relationship between the response and predictors. Gravity Law discloses that travel distance has a multiplicative (inverse) relationship with the ‘masses’ of two places. If the population size can be a representative of built environment, the additive relationship will not hold. Previous studies also shows that the effect sizes of built environment with respect of travel are small and complex. There is not sufficient evidence to support or against the linear hypothesis.

5.1.2 Independent Identically Distributed (IID)

Another essential assumption is that random error are Independent Identically Distributed (IID). Random error is also called residual, which refer to the difference between observed yy and fitted ˆy^y. ˆy^y are the linear combinations of predictors XX. residuals represent the part can not be explained by the model.

e=yˆy

The expected value, the variances, and the covariances among the random errors are the first- and second-moment of residuals. ‘Identical’ means that random errors should have zero mean and constant variance. The homogeneity of variance is also called homoscedasticity.

E(ε)=0,Var(ε)=σ2

‘Independent’ requires the random errors are uncorrelated. That is

Cov[εi,εj]=0,ij

Once the conditions of IID are satisfied, the Gauss - Markov theorem 5.1 proves that least-square method could give the minimum-variance unbiased estimators (MVUE) or called the best linear unbiased estimators (BLUE). These conditions are not strict and make regression method widely applicable.

Theorem 5.1 (Gauss - Markov theorem) For the regression model (1.1) with the assumptions E(ε)=0, Var(ε)=σ2, and uncorrelated errors, the least-squares estimators are unbiased and have minimum variance when compared with all other unbiased estimators that are linear combinations of the yi. (Montgomery, Peck, and Vining 2021)

Another version is that: Under Models II - VII, if λβ is estimable and ˆβ is any solution to the normal equations, then λˆβ is a linear unbiased estimator of λβ and, under Model II, the variance of λˆβ is uniformly less than that of any other linear unbiased estimator of λβ (IX, Theorem E13, p38)

Unfortunately, many of the predictors are correlated. Moreover, the observations from various cities, regions, or counties are very unlikely identical. This issue is called heteroscedasticity. Related contents are in Section of Diagonusis and Validation.

5.1.3 Normality

When conducting hypothesis test and confidence intervals, the required assumption is y|xN(Xβ,σ2I). Maximum Likelihood Methods also requires this assumption.

Evidence has demonstrated that travel distance is not Normal distributed. The Zipf’s law also prove that travel distance follows a power distribution. Using logarithm transformations, the skewed distribution can be converted to an approximate normal distribution.

There are some quantitative methods which can examine nomalirty of the transformed distributions.

5.2 Estimations

5.2.1 Least Squares

  • Ordinary Least Squares

Least-Squares method can be used to estimate the coefficients β in equation (1.1) The dimension of X is n×p, which means the data contain n observations and p1 predictors. The p×1 vector of least-squares estimators is denoted as ˆβ and the solution to the normal equations is

ˆβ=(XX)1Xy

and

ˆσ2=1np(yXˆβ)(yXˆβ)

Here requires XX are invertible, that is, the covariates are linearly independent if X has rank p (Kim 2020, V., Definition, p.22).

Given the estimated coefficients, the model can give the fitted values of response as:

ˆy=Xˆβ=X(XX)1Xy=Hy

where H=X(XX)1X is hat matrix and e=yˆy=yXˆβ=(IH)y

  • Generalized Least Squares

When the observations are not independent or have unequal variances, the covariance matrix of error is not identity matrix. The assumption of regression model V[ε]=σ2I doesn’t hold. Denote V is a known n×n positive definite matrix and V[ε]=σ2V. Then, there exists an n×n symmetric matrix K with rank n and V=KK. Let

z=Ky, B=K1X,and η=Kε

The linear model becomes z=Bβ+η and V[η]=σ2I. If the model is full rank, that is rank(X)=p then XV1X is invertible and the generalized least squares solution is

ˆβGLS=(XV1X)1XV1y

and

ˆσ2GLS=1np(yXˆβGLS)V1(yXˆβGLS)

5.2.2 Standardized coefficients

The value of ˆβj means that, given all other coefficients fixed, for each change of one unit in xj, the average change in the mean of Y. However, the units of predictors X are very different. Hence, the values of coefficients are not comparable.

Unit normal scaling or Unit length scaling can convert ˆβj to dimensionless regression coefficient, which is called standardized regression coefficients. Let

zij=xijˉxjni=1(xijˉxj)2y0i=yiˉyni=1(yiˉy)2

ˆb=(ZZ)1Zy0, orˆbj=ˆβjni=1(xijˉxj)2ni=1(yiˉy)2, j=1,2,...(p1), andˆβ0=ˉyp1j=1ˆβjˉxj

Note that ZZ correlations matrix.

ZZ=[1r12r13r1kr211r23r2kr31321r3krk1rk2k31],Zy0=[r1yr2yr3yrky]

where

rij=nu=1(xuiˉxi)(xujˉxj)nu=1(xuiˉxi)2nu=1(xujˉxj)2rjy=nu=1(xujˉxj)(yuˉy)nu=1(xujˉxj)2nu=1(yuˉy)2

where rij is the simple correlation between xi and xj. rjy is the simple correlation between xj and y

It seems that standardized regression coefficients are comparable. However, the value of ˆbj depends on other predictors. Therefore, comparison between different models is still problematic.

5.2.3 Elasticity

Elasticity is commonly used to determine the relative importance of a variable in terms of its influence on a dependent variable. It is generally interpreted as the percent change in the dependent variable induced by a 1% change in the independent variable (McCarthy 2001).

ei=βiXiYiYiXiXiYi

Table 5.1: Elasticity Estimates for Various Functional Forms
Model Marginal.Effects Elasticity
Linear β βXiYi
Log-linear βYi βXi
Linear-log β1Xi β1Yi
Log-log βYiXi β
Logit βpi(1pi) βXi(1pi)
Poisson βλi βXi
NB βλi βXi

It might be a typo that Reid Ewing and Cervero (2010) use the formula of βˉX(1ˉYn) for Logit model. In Poisson model and Negative Binomial model, λi=exp[xiβ] (Greene, 2018, eq.18-17,21). For truncated Poisson model: δi=(1Pi,0λiPi,0)(1Pi,0)2λiβ (Greene, 2018, eq.18-23). Hurdle model will give separate marginal(partial) effects (Greene, 2018, example 18.20)

5.2.4 Combined effects?

Some studies sums up the standardized coefficients or elasticities and called the summation as combined effects [leeComparingImpactsLocal2020]. Although these values are dimensionless, this method is problematic because different model specifications and data ranges are not comparable.

5.3 Inference

5.3.1 Analysis of Variance

Analysis of Variance (ANOVA) is the fundamental approach in regression analysis. Actually, this method analysis the variation in means rather than variances themselves (Casella & Berger, 2002, Ch.11).

Once the linear relationship holds, the response y can be decomposed to

yy=yHy+y(IH)yyy=ˆβXy+yyˆβXyyynˉy2=ˆβXynˉy2+yyˆβXy(yˉy)2=(ˆyˉy)2+(yˆy)2SST=SSR+SSE

where SST is Sum of Squares Total, SSR is Sum of Squares Regression, and SSE is Sum of Square Error. SSE=ee represents the unknown part of model.

For Generalized Least Squares method, SST=yV1y, SSR=ˆβBz=yV1X(XV1X)1XV1y, and SSE=SSTSSR

5.3.2 Hypothesis Test

  • Significance of Regression

Significance of regression means if the linear relationship between response and predictors is adequate. The hypotheses for testing model adequacy are

H0:β0=β1==βp1=0H1:at least one βj0, j=0,1,...,(p1)

By Theorem D14 (XX, p.90), if an n×1random vector yN(μ,I), then

yyχ2(n,12μμ)

Recall the assumption of y|xN(Xβ,σ2I).
By the additive property of χ2 distribution,

MSEσ2=y(IH)y(np)σ2χ2(np)MSRσ2=yHy(p1)σ2χ2(p1)

Though σ2 is usually unknown, by the relationship between χ2 and F distributions,

F0=MSEMSRF(p1),(np),λ

where λ is the non-centrality parameter. It allows to test the hypotheses given a significance level α. If test statistic F0>Fα,(p1),(np), then one can reject H0.

If a VMT-urban form model added many predictors but adjusted R2 is still low, the association between travel distance and built environment might be spurious.

  • Significance of Coefficients

For testing a specific coefficient, the hypothesis is

H0:βj=0H1:βj0

ˆβ is a linear combination of y. Based on the assumption of y|xN(Xβ,σ2I), it can be proved that ˆβN(β,σ2(XX)1) and

t0=ˆβjse(ˆβj)=ˆβjˆσ2Cjjt(np)

where Cjj is the element at the j row and j column of (XX)1. If |t0|<tα/2,(np), then the test failed to reject the H0, this predictor can be removed from the model. This test is called partial or marginal test because the test statistic for βj depends on all the predictors in the model.

5.3.3 Confidence Intervals

Above results can also construct the confidence interval for each coefficient. A 100(1α) confidence interval for βj is

ˆβjtα/2,(np)ˆσ2Cjjβjˆβj+tα/2,(np)ˆσ2Cjj

References

Casella, George, and Roger L. Berger. 2002. Statistical Inference. 2nd ed. Australia ; Pacific Grove, CA: Thomson Learning.
———. 2010. “Travel and the Built Environment.” Journal of the American Planning Association 76 (3): 265–94. https://doi.org/10.1080/01944361003766766.
Kim, Jong Sung. 2020. “Lecture Notes.”
McCarthy, Patrick S. 2001. Transportation Economics: Theory and Practice : A Case Study Approach. Blackwell Publishers. https://books.google.com?id=NvywQgAACAAJ.
Montgomery, Douglas C., Elizabeth A. Peck, and G. Geoffrey Vining. 2021. Introduction to Linear Regression Analysis. John Wiley & Sons. https://books.google.com?id=tCIgEAAAQBAJ.
Ravishanker, Nalini, and Dipak K. Dey. 2020. A First Course in Linear Model Theory. CRC Press. https://books.google.com?id=i2r0DwAAQBAJ.