Sec 4 Modeling dependence
Easiest to model dependence in stationary case
4.1 Stationary
<Def> stationary: dependence does not change with time
<Def> strictly stationary
\[(X_{t_1}, X_{t_2},...,X_{t_k})\overset{d}{=}(X_{t_1+h}, X_{t_2+h}, ...,X_{t_k+h})\]
- joint probability distribution does not change with time
- \(k=1\)
- means are all identical if means exist
(rule out (排除) trend, seasonality)
- variances are all identical if variances exist
(rule out heteroskedasticity)
- \(k=2\)
-
if variance exist
- \(k\geq3\), get increasingly complicated
- e.g. If \({X_t}\) is i.i.d. then \({X_t}\) is strictly stationary
- strictly stationary is a very strong assumption
<Def> weakly stationary
also known as “stationary”, “covariance stationary”, “second order stationary”
- \(Var(X_t)<\infty\)
- \(E(X_t)\) does not depend on t
- \(Cov(X_t, X_{t+h})\) does not depend on t
- normally depends on \(h\)
- first and second order moment properties do not change with time
- implies all means, variances, covariances exist
- implies means are identical / constant
(rule out trend, seasonality) - with \(h=0\)
\(Cov(X_t, X_{t+h})=Var(X_t)\)
implies variance are constant
(rule out heteroskedasticity)
if there is trend, seasonality or heteroskedasticity in time series, this time series is not stationary
4.2 Weakly Stationary
need to check:
1. \(Var(X_t)<\infty\)
2. \(E(X_t)\) does not depend on t
3. \(Cov(X_t, X_{t+h})\) does not depend on t
e.g.1: \(\{X_t\}\) is strictly stationary and \(Var(X_t)<\infty\) \(\Rightarrow\) weakly stationary
e.g.2:strictly stationary \(\nLeftrightarrow\) weakly stationary
-
\(\{X_t\}\) independent,
\(X_t\sim N(0,1)\) for t odd,
\(X_t=\pm1\) with prob. 0.5 for t even. (i.e. discrete uniform {-1,1})
\[ E(X_t)=0 \text{ for t odd}\\ E(X_t)=1\cdot0.5+(-1)\cdot0.5=0\text{ for t even}\\ \therefore E(X_t)\text{ does not depend on t} \]
\[ Var(X_t)=1\text{ for t odd}\\ \begin{array}{ll}Var(X_t)&=E(X_t^2)-[E(X_t)]^2\\&=1^2\cdot0.5+(-1)^2\cdot0.5-0^2=1\text{ for t even}\end{array}\\ \therefore Var(X_t)<\infty \]
\[ Cov(X_t,X_{t+h})=\big\{\begin{array}{ll} 1, &h=0\\ 0, &h\neq0 \end{array} \text{ does not depend on t}\\ \color{red}{\because\{X_t\}\text{ are indep. and }Var(X_t)<\infty}\\ \therefore\{X_t\}\text{ is weakly stationary} \]
4.3 White Noise (WN)
A sequence \(\{W_t\}\) is called white noise process if each value in the sequence has
- \(E(W_t)=0\)
- \(Var(W_t)=σ^2\;\;∀t\)
- \(Cov(W_t,W_s)=0\;\;if\;\;t≠s\)
Assume the error term is:
- i.i.d. with normal distribution in regression
- WN in time series
e.g.3:
-
\(Z_t\sim WN(0,1)\)
\(X_t=Z_t-0.5Z_{t-1}\) MA(1)
MA: moving average
MA(1): \(X_t=Z_t+a_1Z_{t-1}\)
MA(2): \(X_t=Z_t+a_1Z_{t-1}+a_2Z_{t-2}\)
- \[ \begin{array}{ll} Var(X_t)&=Var(Z_t-0.5Z_{t-1})\\ &=Var(Z_t)+(-0.5)^2Var(Z_{t-1})+2\cdot1\cdot(-0.5)\cdot Cov(Z_t,Z_{t-1}) \color{red}{\because WN\therefore Cov(Z_t,Z_{t-1})=0}\\ &=1+0.25+0=1.25<\infty \end{array} \]
\[ X,Y: r.v.'s \hspace{1cm} A,B: constants\\ Var(AX+BY)=A^2Var(X)+B^2Var(Y)+2ABCov(X,Y) \]
\[ \begin{array}{ll} E(X_t)&=E(Z_t-0.5Z_{t-1})\\ &=E(Z_t)-0.5E(Z_{t-1})=0-0=0\text{ does not depend on t} \end{array} \]
\[ \begin{array}{ll} Cov(X_t,X_{t+h})&=Cov(Z_t-0.5Z_{t-1},Z_{t+h}-0.5Z_{t+h-1})\\ &=\Bigg\{\begin{array}{ll} 1.25, &h=0\\ -0.5, &h=\pm1\\ 0, &o.w. \end{array}\text{ does not depend on t} \end{array}\\ \therefore\{X_t\}\text{ is stationary} \]
4.4 Random Walk Process
<Def> random walk process \[X_t=X_{t-1}+Z_t, \hspace{1cm}Z_t\sim WN(0, \sigma^2)\]
- \(X_t\): price on time \(t\)
- \(X_{t-1}\): price on time \(t-1\)
- \(Z_t=X_t-X_{t-1}\)
- If \(Z_t>0\), price \(\uparrow\)
- If \(Z_t<0\), price \(\downarrow\)
Assume \(X_0=0\) \[ \begin{array}{ll} Var(X_t)&=Var(X_{t-1}+Z_t)\\ &=Var[(X_{t-2}+Z_{t-1})+Z_t]\\ &\vdots\\ &=Var(X_0+Z_1+Z_2+\cdots+Z_t)\\ &=Var\left(\sum_{j=1}^{t}Z_j\right)=\sum_{j=1}^{t}Var(Z_j)=t\cdot\sigma^2\text{ depends on t} \end{array}\\ \therefore\{X_t\}\text{ is stationary} \]