{
  "aliases": null,
  "backlinks": [
    {
      "field": "uses",
      "target": "QQZPWDWS29"
    }
  ],
  "body": "\n## Statement\n\nLet $\\set{P_\\theta : \\theta \\in \\Theta}$ be a [[parametric model|MSC0MR8VMH]] for an observation $X$, and let $g \\colon \\Theta \\to \\R$.\nLet $T = T(X)$ be a [[sufficient statistic|WP4TGXJQJG]] for $\\theta$.\nLet $\\delta(X)$ be an [[estimator|0SSS6ERQ0S]] of $g(\\theta)$ with $E_\\theta[\\delta(X)^2] \u003c \\infty$ for every $\\theta \\in \\Theta$.\nPut $\\delta_1 = E[\\delta(X) \\mid T]$, the [[conditional expectation|0TTDGTEFMN]] given $T$.\nThen:\n\n1. $\\delta_1$ does not depend on $\\theta$, so it is an estimator, and $E_\\theta[\\delta_1] = E_\\theta[\\delta(X)]$ for every $\\theta$.\n2. For every $\\theta \\in \\Theta$, the [[mean squared errors|05NZ1HHQMV]] satisfy\n$$E_\\theta\\big[(\\delta_1 - g(\\theta))^2\\big] \\le E_\\theta\\big[(\\delta(X) - g(\\theta))^2\\big].$$\n3. Equality holds in 2 for a given $\\theta$ if and only if $\\delta(X) = \\delta_1$ with $P_\\theta$-probability $1$.\n\n## Remarks\n\nSufficiency means that the conditional law of $X$ given $T$ is the same for all $\\theta$, so $\\delta_1$ is computed without knowing $\\theta$.\nThe tower property gives the same mean, and the law of total variance gives $\\operatorname{Var}_\\theta(\\delta) = \\operatorname{Var}_\\theta(\\delta_1) + E_\\theta[\\operatorname{Var}(\\delta \\mid T)]$.\nThe last term is nonnegative, and it is zero exactly when $\\delta$ is already a function of $T$ almost surely.\nThe same argument with the conditional Jensen inequality shows that $\\delta_1$ is at least as good as $\\delta$ for every loss that is convex in the estimate.\n\nExample: let $X_1, \\dots, X_n$ be independent Bernoulli variables with parameter $p$.\nThe estimator $\\delta = X_1$ is unbiased for $p$, and $T = X_1 + \\cdots + X_n$ is sufficient.\nBy symmetry $E[X_1 \\mid T] = T/n$, so the Rao–Blackwell step turns $X_1$, of variance $p(1-p)$, into the sample mean, of variance $p(1-p)/n$.\n\nSufficiency is needed: conditioning on a statistic that is not sufficient gives a quantity that depends on the unknown $\\theta$ in general, and so it is not an estimator.\n\n## Explanations\n\n### L1 human\n\nStart with any rough estimate, then average it over all the data sets that carry the same summary information.\nThe averaged estimate is never worse, in mean squared error, than the rough one.\nSo a good estimator should only depend on the data through a sufficient statistic.\n\n### L2 human\n\nThe theorem is the main tool for improving unbiased estimators: conditioning on a sufficient statistic keeps the bias and lowers the variance.\nCombined with completeness of the sufficient statistic, it gives the Lehmann–Scheffé theorem, which identifies the unique minimum-variance unbiased estimator.\nIn practice one often guesses a crude unbiased estimator, such as a single observation or an indicator, and computes its conditional expectation given the sufficient statistic.\n",
  "commit": "ff8a40749dc16545e3d2fc637f6b0196e81d0bfa",
  "front": {
    "about": [
      "WP4TGXJQJG",
      "0SSS6ERQ0S"
    ],
    "kind": "statement",
    "msc": [
      "62F10",
      "62B05"
    ],
    "name": "Rao–Blackwell theorem",
    "role": "theorem",
    "same_as": {
      "wikidata": "Q579515",
      "wikipedia": "Rao–Blackwell theorem"
    },
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/Rao–Blackwell_theorem"
      }
    ],
    "tag": "Y3TX4J41TP",
    "uses": [
      "MSC0MR8VMH",
      "WP4TGXJQJG",
      "0SSS6ERQ0S",
      "0TTDGTEFMN",
      "05NZ1HHQMV"
    ]
  },
  "kind": "statement",
  "link": "https://tamnd.github.io/math2/node/Y3TX4J41TP/",
  "links": [
    {
      "field": "about",
      "target": "WP4TGXJQJG"
    },
    {
      "field": "about",
      "target": "0SSS6ERQ0S"
    },
    {
      "field": "uses",
      "target": "MSC0MR8VMH"
    },
    {
      "field": "uses",
      "target": "WP4TGXJQJG"
    },
    {
      "field": "uses",
      "target": "0SSS6ERQ0S"
    },
    {
      "field": "uses",
      "target": "0TTDGTEFMN"
    },
    {
      "field": "uses",
      "target": "05NZ1HHQMV"
    }
  ],
  "markdown": "https://tamnd.github.io/math2/node/Y3TX4J41TP.md",
  "msc": [
    "62F10",
    "62B05"
  ],
  "name": "Rao–Blackwell theorem",
  "prov": null,
  "role": "theorem",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/Y3T/Y3TX4J41TP.md",
  "status": null,
  "tag": "Y3TX4J41TP",
  "uri": "m2://Y3TX4J41TP",
  "url": "https://tamnd.github.io/math2/wiki/rao-blackwell-theorem/",
  "writing": {
    "tag": "Y3TX4J41TP",
    "kind": "statement",
    "level": "W0",
    "next": [
      {
        "what": "an inventory row",
        "note": "the row of the inventory that the node accounts for",
        "have": 0,
        "want": 1
      },
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      }
    ]
  }
}