Wald's equation

Remarks

Write $S_N = \sum_{i \ge 1} X_i \mathbf{1}_{\set{N \ge i}}$. The event $\set{N \ge i}$ is the complement of $\set{N \le i - 1}$, so it depends only on $X_1, \dots, X_{i-1}$ and is independent of $X_i$. Hence $E[X_i \mathbf{1}_{\set{N \ge i}}] = E[X_1] \, P(N \ge i)$, and summing over $i$ gives $E[X_1] \sum_i P(N \ge i) = E[X_1] \, E[N]$. The same computation with $\abs{X_i}$ shows that the series converges absolutely, which justifies exchanging sum and expectation.

The hypothesis $E[N] < \infty$ is necessary. For the simple symmetric random walk, with $P(X_i = 1) = P(X_i = -1) = 1/2$, let $N$ be the first $n$ with $S_n = 1$. Then $N$ is finite with probability one and $S_N = 1$, so $E[S_N] = 1$, while $E[X_1] = 0$; here $E[N] = \infty$.

Going deeper

For a student

Wald's equation is a key tool in sequential analysis, where it gives the expected sample size of the sequential probability ratio test from the drift of the log-likelihood ratio. It is also used in renewal theory and queueing theory. It can also be derived from the optional stopping theorem applied to the martingale $S_n - n E[X_1]$.