{
  "aliases": null,
  "backlinks": [
    {
      "field": "uses",
      "target": "7SC9E65XDN"
    }
  ],
  "body": "\n## Statement\n\nLet $X$ be [[multivariate normal|0XN3K7RSZY]] on $\\R^p$ with unknown mean $\\theta$ and covariance matrix $\\sigma^2 I_p$, where $\\sigma^2 \u003e 0$ is known.\nLet $g = (g_1, \\dots, g_p) \\colon \\R^p \\to \\R^p$ be [[weakly differentiable|W8W9AP79V5]], with $E_\\theta \\norm{g(X)}^2 \u003c \\infty$ and $E_\\theta \\abs{\\partial_i g_i(X)} \u003c \\infty$ for $i = 1, \\dots, p$.\nConsider the estimator $\\delta(X) = X + g(X)$ of $\\theta$, and put\n$$\\mathrm{SURE}(X) = p\\sigma^2 + \\norm{g(X)}^2 + 2\\sigma^2 \\sum_{i=1}^{p} \\partial_i g_i(X).$$\nThen $\\mathrm{SURE}(X)$ is an [[unbiased|0BWGTN92FG]] estimator of the [[risk|N26TSVRH2S]] of $\\delta$ under squared error loss:\n$$E_\\theta\\big[ \\mathrm{SURE}(X) \\big] = E_\\theta \\norm{\\delta(X) - \\theta}^2.$$\n\n## Remarks\n\nExpand $\\norm{X - \\theta + g(X)}^2 = \\norm{X - \\theta}^2 + \\norm{g(X)}^2 + 2 \\sum_i (X_i - \\theta_i) g_i(X)$.\nThe first term has expectation $p\\sigma^2$.\nBy the multivariate [[Stein's lemma|6PWC83R457]], $E_\\theta[(X_i - \\theta_i) g_i(X)] = \\sigma^2 E_\\theta[\\partial_i g_i(X)]$, which gives the formula.\n\nThe point is that $\\mathrm{SURE}(X)$ does not involve the unknown $\\theta$.\nSo it can be computed from the data and minimized over a family of estimators, for example to choose a threshold or a smoothing parameter.\n\nExample: the [[James–Stein estimator|709T196XCT]] with $\\sigma = 1$ corresponds to $g(x) = -(p - 2) x / \\norm{x}^2$.\nThen $\\mathrm{SURE}(X) = p - (p - 2)^2 / \\norm{X}^2$, whose expectation is the risk of $\\delta_{JS}$.\nIt is below $p$ for every $X$, which shows that $\\delta_{JS}$ has risk below $p$ when $p \\ge 3$.\n\n## Explanations\n\n### L1 human\n\nUsually the error of an estimate cannot be computed, because the truth is unknown.\nFor normal data with known noise level, Stein found a formula built only from the data whose average equals the average squared error.\nOne can therefore compare estimators, or tune them, using the data alone.\n\n### L2 human\n\nThe estimate is the standard tool to prove that shrinkage estimators beat the raw observation, as in Stein's example.\nIt is also used in practice to select tuning parameters, for instance the threshold in wavelet shrinkage, as an alternative to cross-validation.\nThe derivative term $\\sum_i \\partial_i g_i$ is the divergence of $g$.\nThe expected divergence of $\\delta$, which is $p$ plus the expected divergence of $g$, is the effective number of degrees of freedom used by the estimator.\n",
  "commit": "ff8a40749dc16545e3d2fc637f6b0196e81d0bfa",
  "front": {
    "about": [
      "N26TSVRH2S"
    ],
    "kind": "statement",
    "msc": [
      "62C15",
      "62F10",
      "62J07"
    ],
    "name": "Stein's unbiased risk estimate",
    "role": "theorem",
    "same_as": {
      "wikidata": "Q7606742",
      "wikipedia": "Stein's unbiased risk estimate"
    },
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/Stein's_unbiased_risk_estimate"
      }
    ],
    "tag": "30XVCMK916",
    "uses": [
      "0XN3K7RSZY",
      "6PWC83R457",
      "N26TSVRH2S",
      "05NZ1HHQMV",
      "0BWGTN92FG",
      "W8W9AP79V5",
      "709T196XCT"
    ]
  },
  "kind": "statement",
  "link": "https://tamnd.github.io/math2/node/30XVCMK916/",
  "links": [
    {
      "field": "about",
      "target": "N26TSVRH2S"
    },
    {
      "field": "uses",
      "target": "0XN3K7RSZY"
    },
    {
      "field": "uses",
      "target": "6PWC83R457"
    },
    {
      "field": "uses",
      "target": "N26TSVRH2S"
    },
    {
      "field": "uses",
      "target": "05NZ1HHQMV"
    },
    {
      "field": "uses",
      "target": "0BWGTN92FG"
    },
    {
      "field": "uses",
      "target": "W8W9AP79V5"
    },
    {
      "field": "uses",
      "target": "709T196XCT"
    }
  ],
  "markdown": "https://tamnd.github.io/math2/node/30XVCMK916.md",
  "msc": [
    "62C15",
    "62F10",
    "62J07"
  ],
  "name": "Stein's unbiased risk estimate",
  "prov": null,
  "role": "theorem",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/30X/30XVCMK916.md",
  "status": null,
  "tag": "30XVCMK916",
  "uri": "m2://30XVCMK916",
  "url": "https://tamnd.github.io/math2/wiki/steins-unbiased-risk-estimate/",
  "writing": {
    "tag": "30XVCMK916",
    "kind": "statement",
    "level": "W0",
    "next": [
      {
        "what": "an inventory row",
        "note": "the row of the inventory that the node accounts for",
        "have": 0,
        "want": 1
      },
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      }
    ]
  }
}