---
tag: 6PWC83R457
kind: statement
role: lemma
name: Stein's lemma
msc: [62C15, "62E10", "60E05"]
uses: [05AQE0QR0D, 0WQNRCNY3M, ZT6F4C2YHK, 08Y1BBQTK9, 0XN3K7RSZY]
about: [05AQE0QR0D]
sources:
  - url: https://en.wikipedia.org/wiki/Stein's_lemma
same_as:
  wikidata: Q7606741
  wikipedia: Stein's lemma
---

## Statement

Let $Z$ be a [[normal|05AQE0QR0D]] random variable with mean $\mu \in \R$ and variance $\sigma^2 > 0$.
Let $g \colon \R \to \R$ be [[absolutely continuous|ZT6F4C2YHK]] on every bounded interval, with derivative $g'$, and assume $E\abs{g'(Z)} < \infty$.
Then $E\abs{g(Z)(Z - \mu)} < \infty$ and
$$E\big[ g(Z) (Z - \mu) \big] = \sigma^2 \, E\big[ g'(Z) \big].$$

## Remarks

The normal density $\varphi$ with mean $\mu$ and variance $\sigma^2$ satisfies $\varphi'(z) = -\frac{z - \mu}{\sigma^2} \varphi(z)$.
So the left side is $-\sigma^2 \int g(z) \varphi'(z) \, dz$, and [[integration by parts|08Y1BBQTK9]] moves the derivative onto $g$.
The integrability hypothesis is what controls the boundary terms at $\pm\infty$; the careful proof writes $g(z) - g(\mu)$ as an integral of $g'$ and uses Fubini's theorem.

Example: with $\mu = 0$ and $g(z) = z^3$, the left side is $E[Z^4] = 3\sigma^4$.
The right side is $\sigma^2 E[3Z^2] = 3\sigma^4$, as it should be.

The multivariate form is used in estimation.
If $X$ is [[multivariate normal|0XN3K7RSZY]] with mean $\theta$ and covariance $\sigma^2 I_p$, then $E[(X_i - \theta_i) g(X)] = \sigma^2 E[\partial_i g(X)]$ for suitable $g \colon \R^p \to \R$.

## Explanations

### L1 human

For a bell-shaped random variable, the average of $g(Z)$ times the deviation of $Z$ from its mean equals the variance times the average slope of $g$.
It is an integration by parts in disguise.
It turns covariances with the data into averages of derivatives, which are often easier to handle.

### L2 human

The identity characterizes the normal distribution: if it holds for all bounded smooth $g$, then $Z$ is normal with mean $\mu$ and variance $\sigma^2$.
This is the starting point of Stein's method for proving normal approximations.
In decision theory, the multivariate form gives Stein's unbiased risk estimate and through it the risk of the James–Stein estimator.