Instrumental variables estimation

Remarks

The name two-stage least squares comes from its computation. First regress each column of $X$ on $Z$, which gives $\hat X = P_Z X$, the orthogonal projection of $X$ onto the column space of $Z$. Then regress $y$ on $\hat X$ by ordinary least squares. Under clauses 1 and 2, for independent and identically distributed observations with finite second moments, $\hat\beta_{\mathrm{IV}}$ is consistent for $\beta$, while ordinary least squares in general is not.

Example: $k = L = 1$, no intercept, $y_i = \beta x_i + \varepsilon_i$. Exogeneity gives $E[z_i y_i] = \beta E[z_i x_i]$, so $\beta = E[z_i y_i] / E[z_i x_i]$. The estimator $\sum_i z_i y_i / \sum_i z_i x_i$ is the sample analogue.

Non-example: if $z_i$ is independent of $x_i$ and $E[z_i] = 0$, then $E[z_i x_i] = 0$ and clause 2 fails; the ratio above has a denominator tending to $0$ and does not identify $\beta$. Instruments that satisfy clause 2 only barely, called weak instruments, give unreliable estimates in finite samples.