---
tag: 30XVCMK916
kind: statement
role: theorem
name: Stein's unbiased risk estimate
msc: [62C15, 62F10, 62J07]
uses: [0XN3K7RSZY, 6PWC83R457, N26TSVRH2S, 05NZ1HHQMV, 0BWGTN92FG, W8W9AP79V5, 709T196XCT]
about: [N26TSVRH2S]
sources:
  - url: https://en.wikipedia.org/wiki/Stein's_unbiased_risk_estimate
same_as:
  wikidata: Q7606742
  wikipedia: Stein's unbiased risk estimate
---

## Statement

Let $X$ be [[multivariate normal|0XN3K7RSZY]] on $\R^p$ with unknown mean $\theta$ and covariance matrix $\sigma^2 I_p$, where $\sigma^2 > 0$ is known.
Let $g = (g_1, \dots, g_p) \colon \R^p \to \R^p$ be [[weakly differentiable|W8W9AP79V5]], with $E_\theta \norm{g(X)}^2 < \infty$ and $E_\theta \abs{\partial_i g_i(X)} < \infty$ for $i = 1, \dots, p$.
Consider the estimator $\delta(X) = X + g(X)$ of $\theta$, and put
$$\mathrm{SURE}(X) = p\sigma^2 + \norm{g(X)}^2 + 2\sigma^2 \sum_{i=1}^{p} \partial_i g_i(X).$$
Then $\mathrm{SURE}(X)$ is an [[unbiased|0BWGTN92FG]] estimator of the [[risk|N26TSVRH2S]] of $\delta$ under squared error loss:
$$E_\theta\big[ \mathrm{SURE}(X) \big] = E_\theta \norm{\delta(X) - \theta}^2.$$

## Remarks

Expand $\norm{X - \theta + g(X)}^2 = \norm{X - \theta}^2 + \norm{g(X)}^2 + 2 \sum_i (X_i - \theta_i) g_i(X)$.
The first term has expectation $p\sigma^2$.
By the multivariate [[Stein's lemma|6PWC83R457]], $E_\theta[(X_i - \theta_i) g_i(X)] = \sigma^2 E_\theta[\partial_i g_i(X)]$, which gives the formula.

The point is that $\mathrm{SURE}(X)$ does not involve the unknown $\theta$.
So it can be computed from the data and minimized over a family of estimators, for example to choose a threshold or a smoothing parameter.

Example: the [[James–Stein estimator|709T196XCT]] with $\sigma = 1$ corresponds to $g(x) = -(p - 2) x / \norm{x}^2$.
Then $\mathrm{SURE}(X) = p - (p - 2)^2 / \norm{X}^2$, whose expectation is the risk of $\delta_{JS}$.
It is below $p$ for every $X$, which shows that $\delta_{JS}$ has risk below $p$ when $p \ge 3$.

## Explanations

### L1 human

Usually the error of an estimate cannot be computed, because the truth is unknown.
For normal data with known noise level, Stein found a formula built only from the data whose average equals the average squared error.
One can therefore compare estimators, or tune them, using the data alone.

### L2 human

The estimate is the standard tool to prove that shrinkage estimators beat the raw observation, as in Stein's example.
It is also used in practice to select tuning parameters, for instance the threshold in wavelet shrinkage, as an alternative to cross-validation.
The derivative term $\sum_i \partial_i g_i$ is the divergence of $g$.
The expected divergence of $\delta$, which is $p$ plus the expected divergence of $g$, is the effective number of degrees of freedom used by the estimator.