Stein's lemma

Remarks

The normal density $\varphi$ with mean $\mu$ and variance $\sigma^2$ satisfies $\varphi'(z) = -\frac{z - \mu}{\sigma^2} \varphi(z)$. So the left side is $-\sigma^2 \int g(z) \varphi'(z) \, dz$, and integration by parts moves the derivative onto $g$. The integrability hypothesis is what controls the boundary terms at $\pm\infty$; the careful proof writes $g(z) - g(\mu)$ as an integral of $g'$ and uses Fubini's theorem.

Example: with $\mu = 0$ and $g(z) = z^3$, the left side is $E[Z^4] = 3\sigma^4$. The right side is $\sigma^2 E[3Z^2] = 3\sigma^4$, as it should be.

The multivariate form is used in estimation. If $X$ is multivariate normal with mean $\theta$ and covariance $\sigma^2 I_p$, then $E[(X_i - \theta_i) g(X)] = \sigma^2 E[\partial_i g(X)]$ for suitable $g \colon \R^p \to \R$.

Going deeper

For a student

The identity characterizes the normal distribution: if it holds for all bounded smooth $g$, then $Z$ is normal with mean $\mu$ and variance $\sigma^2$. This is the starting point of Stein's method for proving normal approximations. In decision theory, the multivariate form gives Stein's unbiased risk estimate and through it the risk of the James–Stein estimator.