6.4 Example: An $AR(2)$ Model

Auto-regressive models are commonly used in time series analysis and typically are written in the following form,

$$z_t = \theta_1z_{t-1} + \theta_2z_{t-2} + \cdots + \theta_pz_{t-p} + \varepsilon_t$$

where the above model would be denoted an autoregressive model of order $p$ or $AR(p)$. While there are a variety of approaches to fitting $AR(p)$ models proposed, we can easily fit them into the state space model framework and use the Kalman filter machinery to do maximum likelihood estimation.

Consider an $AR(2)$ model of the form

$$z_t = \phi_1z_{t-1} + \phi_2 z_{t-2} + w_t$$ For now, let’s assume that the time series $z_t$ has mean $0$ and $w_t\sim\mathcal{N}(0,\tau^2)$.

We can write the state vector as

$$x_t = \left[\begin{array}{c} z_{t}\\ \phi_2z_{t-1} \end{array} \right]$$

and

$$x_{t-1} = \left[\begin{array}{c} z_{t-1}\\ \phi_2z_{t-2} \end{array} \right]$$

We can furthermore write

$$\Theta = \left[\begin{array}{cc} \phi_1 & 1\\ \phi_2 & 0 \end{array} \right]$$

so that we have

$$\left[\begin{array}{c} z_{t}\\ \phi_2z_{t-1} \end{array} \right] = \left[\begin{array}{cc} \phi_1 & 1\\ \phi_2 & 0 \end{array} \right] \left[\begin{array}{c} z_{t-1}\\ \phi_2z_{t-2} \end{array} \right] + \left[\begin{array}{c} 1\\ 0 \end{array} \right] w_t$$ which follows the form

$$x_t = \Theta x_{t-1} + B w_t$$

Although it seems weird and totally artificial, it does work. Sometimes you have to take a step back before you take a step forward!

Finally, the observation equation is simply

$$y_t = Ax_t$$

where

$$A = [\begin{array}{cc} 1 & 0 \end{array} ]$$

In this case, there is no variance term for the observation equation.

Because this model is slightly different from what we’ve had previously, we need to alter the update step in the Kalman filter. Here, the update step is now

$$\begin{eqnarray*} x_t^{t-1} & = & \Theta x_{t-1}^{t-1}\\ P_t^{t-1} & = & \Theta P_{t-1}^{t-1}\Theta^\prime + \tau^2 BB^\prime \end{eqnarray*}$$

[1] 19000.06

[1]  8.996889e-01 -8.296826e-02  4.528905e+00 -3.030640e-12 -3.914266e-12

Here are the point estimates for the autoregressive parameters.

[1]  0.89968888 -0.08296826

And here are their asymptotic standard errors.

[1] 0.03675802 0.03676557