Stein's example

Counterexample

Let $p \ge 3$ and let $X$ have the multivariate normal distribution on $\R^p$ with unknown mean $\theta \in \R^p$ and covariance matrix the identity $I_p$. Estimate $\theta$ under the loss $L(\theta, a) = \norm{a - \theta}^2$, the squared Euclidean distance.

The estimator $\delta_0(X) = X$ has risk $R(\theta, \delta_0) = E_\theta \norm{X - \theta}^2 = p$ for every $\theta$. It is not admissible. The James–Stein estimator $$\delta_{JS}(X) = \Big( 1 - \frac{p - 2}{\norm{X}^2} \Big) X$$ has risk $$R(\theta, \delta_{JS}) = p - (p - 2)^2 \, E_\theta\Big[ \frac{1}{\norm{X}^2} \Big].$$ For $p \ge 3$ the expectation is finite and positive, so $R(\theta, \delta_{JS}) < p$ for every $\theta$. Thus $\delta_{JS}$ dominates $X$.

Check of the risk formula: write $\delta_{JS}(X) = X + g(X)$ with $g(x) = -(p - 2) x / \norm{x}^2$. Then $\norm{g(x)}^2 = (p - 2)^2 / \norm{x}^2$ and $\sum_i \partial_i g_i(x) = -(p - 2)^2 / \norm{x}^2$. Stein's unbiased risk estimate gives $R(\theta, \delta_{JS}) = E_\theta\big[p + \norm{g(X)}^2 + 2 \sum_i \partial_i g_i(X)\big]$, which is the formula above.