Sec 3 Base Definition

3.1 Variance, Covariance, Correlation

<Def> Covariance

\[Cov(X_s, X_t)=E[(X_s-\mu_s)(X_t-\mu_t)]\] <Def> Correlation

\[Corr=\frac{Cov(X,Y)}{\sqrt{Var(X)Var(Y)}}\]

  • Variance of linear combination

\[Var\left(a_0+\sum_{j=1}^pa_jX_j\right)=\sum_{j=1}^p\sum_{k=1}^pa_ja_k\cdot Cov(X_j,X_k)\]

  • Covariance of linear combination

\[Cov\left(a_0+\sum_{j=1}^pa_jX_j, b_0+\sum_{k=1}^qb_kY_k\right)=\sum_{j=1}^p\sum_{k=1}^qa_jb_k\cdot Cov(X_j,Y_k)\]

3.2 Uncorrelated, Independent

  • r.v. with zero correlation \(\Rightarrow\) uncorrelated
  • independent + finite variance \(\Rightarrow\) uncorrelated
  • uncorrelated \(\nRightarrow\) independent

<Def> uncorrelated

\[Corr(X,Y)=0\]

<Def> independent

\[P(X\cap Y)=P(X)P(Y)\]

3.3 Autocovariance, Autocorrelation, Cross-covariance, Cross-correlation

3.3.1 Population

<Def> autocovariance function

\[Cov(X_s, X_t)= \gamma_X(s,t)=E[(X_s-\mu_s)(X_t-\mu_t)]\] \[measuring\:time: s,t=t_1,t_2,...\]

<Def> autocorrelation function (ACF)

\[\rho(s,t)=\frac{\gamma(s,t)}{\sqrt{\gamma(s,s)\gamma(t,t)}}=\frac{Cov(X_s,X_t)}{\sqrt{Var(X_s)Var(X_t)}}\] <Def> cross-covariance function

\[\gamma_{XY}(s,t)=E[(X_s-\mu_{X_s})(Y_t-\mu_{Y_t})]\]

<Def> cross-correlation function (CCF)

\[\rho_{XY}(s,t)=\frac{\gamma_{XY}(s,t)}{\sqrt{\gamma_X(s,s)\gamma_Y(t,t)}}=\frac{Cov(X_s,Y_t)}{\sqrt{Var(X_s)Var(Y_t)}}\]

Covariance: different covatiate Autocovariance: same variable but at different time

3.3.2 Stationary Case

times series \(\{X_t\}\) is stationary

    <Def> autocovariance function \[ \gamma(h)=Cov(X_t, X_{t+h})\\ \color{red}{\gamma(0)=Var(X_t)} \]

    <Def> autocorrelation function (ACF) \[ \rho(h)=Corr(X_t, X_{t+h})=\frac{Cov(X_t, X_{t+h})}{\sqrt{Var(X_t)Var(X_{t+h})}}=\frac{\gamma(h)}{\gamma(0)} \]
two times series \(\{X_t\},\{Y_t\}\) are jointly stationary

    <Def> cross-covariance function \[ \gamma_{XY}(h)=Cov(X_t, Y_{t+h})=E[(X_t-\mu_{X})(Y_{t+h}-\mu_{Y})]\\ \text{ is a function only of lag h} \]

    <Def> cross-correlation function (CCF) \[ \rho_{XY}(h)=Corr(X_t, Y_{t+h})=\frac{\gamma_{XY}(h)}{\sqrt{\gamma_X(0)\gamma_Y(0)}} \]

3.3.3 Sample

<Def> sample autocovariance function \[ \hat\gamma(h)=n^{-1}\sum_{t=1}^{n-h}(X_t-\bar X)(X_{t+h}-\bar X) \]

<Def> sample autocorrelation function (ACF) \[ \hat\rho(h)=\frac{\hat\gamma(h)}{\hat\gamma(0)} \]

<Prop> large sample distribution of ACF \[ \hat\rho_X(h)\sim N(0,\frac{1}{\sqrt{n}})\\ \text{ if }X_t\text{ is }WN\text{ and n large} \] \(H_0: \text{follow WN assumption}\)

  • accept WN assumption, if all ACF are within \(\pm1.96\times\frac{1}{\sqrt n}\)
  • see the follow example 3.1
acf(diff(gtemp), 48)
The ACF plot

Figure 3.1: The ACF plot

<Def> sample cross-covariance function \[ \hat\gamma_{XY}(h)=n^{-1}\sum_{t=1}^{n-h}(X_t-\bar X)(Y_{t+h}-\bar Y) \]

<Def> sample cross-correlation function (CCF) \[ \hat\rho_{XY}(h)=\frac{\hat\gamma_{XY}(h)}{\sqrt{\hat\gamma_X(0)\hat\gamma_Y(0)}} \]

<Prop> large sample distribution of CCF \[ \hat\rho_{XY}(h)\sim N(0,\frac1n)\\ \text{ if at least one process is white independent noise} \]