Hájek–Le Cam convolution theorem

Remarks

Idea of proof: by LAN, the pair $\big(\sqrt{n}(T_n - \theta), \Delta_{n,\theta}\big)$ has a joint limit along subsequences, and Le Cam's third lemma gives its limit under the local alternatives. Regularity says that this limit does not change with $h$; comparing characteristic functions then shows that the limit of $\sqrt{n}(T_n - \theta) - I_\theta^{-1}\Delta_{n,\theta}$ is independent of the limit of $I_\theta^{-1}\Delta_{n,\theta}$, which is $\mathcal{N}_k(0, I_\theta^{-1})$.

Consequence: if $L_\theta$ has covariance $\Sigma$, then $\Sigma - I_\theta^{-1}$ is positive semidefinite, so $I_\theta^{-1}$ is the best possible asymptotic covariance for regular estimators. An estimator with $L_\theta = \mathcal{N}_k(0, I_\theta^{-1})$, such as the maximum likelihood estimator in smooth models, is called asymptotically efficient.

The regularity hypothesis is necessary: there exist superefficient estimators that beat $I_\theta^{-1}$ at single points, but they are not regular there.

Going deeper

For a student

The theorem is the asymptotic counterpart of the Cramér–Rao bound, but it needs no unbiasedness and it describes the whole limit law, not just its variance. "Regular" rules out estimators that are tuned to one parameter value, which could otherwise do better there. Together with the local asymptotic minimax theorem, it makes precise in what sense maximum likelihood estimators are asymptotically optimal in LAN models.