Yule–Walker equations

Remarks

Multiply the defining equation by $X_{t-h}$ and take expectations. For $h \ge 1$, causality makes $X_{t-h}$ a function of $\varepsilon_s$ with $s \le t - h < t$, so $E[\varepsilon_t X_{t-h}] = 0$, which gives the first equations. For $h = 0$, $E[\varepsilon_t X_t] = E[\varepsilon_t^2] = \sigma^2$, which gives the last one. The matrix $\Gamma_p$ is a symmetric Toeplitz matrix, and it is nonsingular because $\gamma(0) > 0$ and $\gamma(h) \to 0$ as $h \to \infty$. So the autocovariances $\gamma(0), \dots, \gamma(p)$ determine $\phi$ and $\sigma^2$.

Example: for $p = 1$, the equations read $\gamma(1) = \phi_1 \gamma(0)$ and $\gamma(0) = \phi_1 \gamma(1) + \sigma^2$. Hence $\phi_1 = \rho(1)$ and $\gamma(0) = \sigma^2 / (1 - \phi_1^2)$.

The Yule–Walker estimator replaces $\gamma$ by the sample autocovariance $\hat\gamma$ and solves $\hat\Gamma_p \hat\phi = \hat\gamma_p$. Causality is needed: for the non-causal stationary solution of $X_t = 2 X_{t-1} + \varepsilon_t$, the shock $\varepsilon_t$ is correlated with the past values $X_{t-h}$ and the first equation fails.

Going deeper

For a student

The equations give the simplest estimator of the coefficients of an AR($p$) model, computed from the sample autocorrelation function. For a Gaussian causal AR process, this estimator has the same asymptotic distribution as the maximum likelihood estimator. The Durbin–Levinson algorithm solves the Toeplitz system for $p = 1, 2, 3, \dots$ in turn, and the last coefficient at each order is the partial autocorrelation. The equations are named after Udny Yule and Gilbert Walker.