## 6.1 Example: An $$AR(1)$$ Model

Consider the following example, which describes and auto-regressive model of order $$1$$,

$y_t=\phi y_{t-1} + w_t$

where $$w_t\sim\mathcal{N}(0,\tau^2)$$ and we assume that $$\mathbb{E}[y_t]=0$$ for all $$t$$. What is the joint distribution of the the $$y_t$$s in this case?

If we assume that the process is 2nd-order stationary then for the marginal variances, we have

$\text{Var}(y_t) = \phi^2\text{Var}(y_{t-1}) + \tau^2.$ However, the stationarity assumption implies that $$\text{Var}(y_t)=\text{Var}(y_{t-1})$$. Therefore, if we rearrange terms, we must have that

$\text{Var}(y_t) = \frac{\tau^2}{1-\phi^2}.$ Note that the expression makes little sense if $$|\phi|\geq 1$$ so from here on we will assume $$|\phi|<1$$. Furthermore, we can then show that

$\begin{eqnarray*} \text{Cov}(y_t,y_{t-1}) & = & \text{Cov}(\phi y_{t-1}, y_{t-1})\\ & = & \phi\text{Var}(y_{t-1})\\ & = & \phi\,\frac{\tau^2}{1-\phi^2} \end{eqnarray*}$ and because of the sequential dependence of the $$y_t$$s on each other, we have

$\text{Cov}(y_t,y_{t-j}) = \phi^{|j|}\,\frac{\tau^2}{1-\phi^2}.$ From all this, we can see that the joint distribution of $$y_1,\dots,y_n$$ is Normal with mean vector $$0$$ and a covariance matrix that whose elements are complex nonlinear functions of $$\phi$$. While it is theoretically possible to compute this joint density and maximize it with respect to $$\phi$$ and $$\tau$$, some challenges arise relatively quickly:

• As $$n$$ increase, the $$n\times n$$ covariance matrix quickly grows in size, making the computations more cumbersome, especially because some form of matrix decomposition must occur.

• As $$n$$ increases, we are taking larger and larger powers of $$\phi$$, which can quickly lead to numerical instability and unfortunately cannot be solved by taking logs.

• The formulation above ignores the sequential structure of the $$AR(1)$$ model, which could be used to simplify the computations.

Thankfully, the Kalman filter provides a computationally efficient way to evaluate this complex likelihood that addresses both of these problems.