{
  "aliases": null,
  "backlinks": [
    {
      "field": "uses",
      "target": "1PKZW82H0C"
    }
  ],
  "body": "\n## Statement\n\nLet $\\set{f(\\cdot\\,; \\theta) : \\theta \\in \\Theta}$ be a family of densities with respect to a fixed measure $\\mu$, where $\\Theta \\subseteq \\R$ is an open interval.\nAssume the following regularity conditions.\n\n1. The set $\\set{x : f(x; \\theta) \u003e 0}$ does not depend on $\\theta$.\n2. For every such $x$, the map $\\theta \\mapsto f(x; \\theta)$ is differentiable.\n3. The [[Fisher information|G1Z3WSTE8Q]] $I(\\theta)$ satisfies $0 \u003c I(\\theta) \u003c \\infty$ for every $\\theta$.\n\nLet $X$ have density $f(\\cdot\\,; \\theta)$, and let $T = T(X)$ be a [[statistic|0Q5CG2NEQ8]] with $E_\\theta[T^2] \u003c \\infty$ for every $\\theta$.\nPut $\\psi(\\theta) = E_\\theta[T]$.\nAssume that differentiation in $\\theta$ may be taken under the integral sign in $\\int f(x; \\theta) \\, \\mu(dx) = 1$ and in $\\int T(x) f(x; \\theta) \\, \\mu(dx) = \\psi(\\theta)$.\nThen $\\psi$ is differentiable, and for every $\\theta \\in \\Theta$,\n$$\\operatorname{Var}_\\theta(T) \\ge \\frac{\\psi'(\\theta)^2}{I(\\theta)}.$$\nIn particular, if $T$ is an [[unbiased|0BWGTN92FG]] [[estimator|0SSS6ERQ0S]] of $\\theta$, then $\\operatorname{Var}_\\theta(T) \\ge 1/I(\\theta)$.\n\n## Remarks\n\nLet $s$ be the [[score|CP97GRVTQ0]].\nDifferentiating the two integrals gives $E_\\theta[s] = 0$ and $E_\\theta[T s] = \\psi'(\\theta)$, so $\\operatorname{Cov}_\\theta(T, s) = \\psi'(\\theta)$.\nThe [[Cauchy–Schwarz inequality|03PGHKAJG3]] then gives $\\psi'(\\theta)^2 \\le \\operatorname{Var}_\\theta(T) \\operatorname{Var}_\\theta(s) = \\operatorname{Var}_\\theta(T) \\, I(\\theta)$.\nFor $n$ independent observations the information is $n I(\\theta)$, so the bound becomes $\\psi'(\\theta)^2 / (n I(\\theta))$.\nIn dimension $d$, the bound reads $\\operatorname{Cov}_\\theta(T) \\succeq D \\, I(\\theta)^{-1} D^{\\mathsf T}$, with $D$ the Jacobian matrix of $\\psi$.\n\nExample: for $n$ independent normal observations with mean $\\theta$ and known variance $\\sigma^2$, one has $I(\\theta) = 1/\\sigma^2$.\nThe sample mean is unbiased with variance $\\sigma^2/n = 1/(n I(\\theta))$, so it attains the bound.\n\nThe fixed support is necessary.\nFor $n$ independent uniform observations on $(0, \\theta)$, the unbiased estimator $\\frac{n+1}{n} \\max_i X_i$ has variance $\\theta^2 / (n(n+2))$.\nThis is smaller than $\\theta^2/n$, the value of the formal bound with $E_\\theta[s^2] = 1/\\theta^2$.\n\n## Explanations\n\n### L1 human\n\nNo unbiased estimate can be more precise than a limit fixed by the model itself.\nThe limit is one over the Fisher information, so models whose data carry much information allow precise estimates.\nAn estimator that reaches the limit cannot be improved among unbiased estimators.\n\n### L2 human\n\nThe bound is the benchmark for the efficiency of estimators: the ratio of the bound to the actual variance measures how much information an estimator wastes.\nIt is attained exactly only in special cases, essentially for exponential families and the natural statistic.\nAsymptotically, under regularity conditions, the maximum likelihood estimator has variance close to $1/(n I(\\theta))$ and so reaches the bound in the limit.\nThe regularity conditions matter: models with a moving support, such as the uniform family, can be estimated faster than the bound allows.\n",
  "commit": "ff8a40749dc16545e3d2fc637f6b0196e81d0bfa",
  "front": {
    "about": [
      "G1Z3WSTE8Q",
      "0SSS6ERQ0S"
    ],
    "kind": "statement",
    "msc": [
      "62F10",
      "62B10"
    ],
    "name": "Cramér–Rao bound",
    "role": "theorem",
    "same_as": {
      "wikidata": "Q1138810",
      "wikipedia": "Cramér–Rao bound"
    },
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/Cramér–Rao_bound"
      }
    ],
    "tag": "1JFKTEQVRX",
    "uses": [
      "G1Z3WSTE8Q",
      "CP97GRVTQ0",
      "0Q5CG2NEQ8",
      "0SSS6ERQ0S",
      "0BWGTN92FG",
      "0477R9PYAE",
      "03PGHKAJG3"
    ]
  },
  "kind": "statement",
  "link": "https://tamnd.github.io/math2/node/1JFKTEQVRX/",
  "links": [
    {
      "field": "about",
      "target": "G1Z3WSTE8Q"
    },
    {
      "field": "about",
      "target": "0SSS6ERQ0S"
    },
    {
      "field": "uses",
      "target": "G1Z3WSTE8Q"
    },
    {
      "field": "uses",
      "target": "CP97GRVTQ0"
    },
    {
      "field": "uses",
      "target": "0Q5CG2NEQ8"
    },
    {
      "field": "uses",
      "target": "0SSS6ERQ0S"
    },
    {
      "field": "uses",
      "target": "0BWGTN92FG"
    },
    {
      "field": "uses",
      "target": "0477R9PYAE"
    },
    {
      "field": "uses",
      "target": "03PGHKAJG3"
    }
  ],
  "markdown": "https://tamnd.github.io/math2/node/1JFKTEQVRX.md",
  "msc": [
    "62F10",
    "62B10"
  ],
  "name": "Cramér–Rao bound",
  "prov": null,
  "role": "theorem",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/1JF/1JFKTEQVRX.md",
  "status": null,
  "tag": "1JFKTEQVRX",
  "uri": "m2://1JFKTEQVRX",
  "url": "https://tamnd.github.io/math2/wiki/cramer-rao-bound/",
  "writing": {
    "tag": "1JFKTEQVRX",
    "kind": "statement",
    "level": "W0",
    "next": [
      {
        "what": "an inventory row",
        "note": "the row of the inventory that the node accounts for",
        "have": 0,
        "want": 1
      },
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      }
    ]
  }
}