{
  "aliases": null,
  "backlinks": null,
  "body": "\n## Definition\n\nLet $\\set{f(\\cdot\\,; \\theta) : \\theta \\in \\Theta}$ be a [[parametric family|MSC0MR8VMH]] of densities with respect to a fixed measure, where $\\Theta \\subseteq \\R$ is an open interval.\nAssume that $\\theta \\mapsto \\log f(x; \\theta)$ is differentiable for every $x$ with $f(x; \\theta) \u003e 0$.\nLet $X$ have density $f(\\cdot\\,; \\theta)$, and let\n$$s(\\theta; X) = \\frac{\\partial}{\\partial \\theta} \\log f(X; \\theta)$$\nbe the [[score|CP97GRVTQ0]].\nThe *Fisher information* at $\\theta$ is\n$$I(\\theta) = E_\\theta\\big[ s(\\theta; X)^2 \\big] \\in [0, \\infty].$$\nFor $\\Theta \\subseteq \\R^d$ open, the *Fisher information matrix* is the $d \\times d$ matrix with entries $I_{ij}(\\theta) = E_\\theta[ s_i(\\theta; X) \\, s_j(\\theta; X) ]$, where $s_i = \\partial \\log f / \\partial \\theta_i$.\n\n## Remarks\n\nUnder the usual regularity conditions, differentiation under the integral sign gives $E_\\theta[s(\\theta; X)] = 0$.\nThen $I(\\theta) = \\operatorname{Var}_\\theta(s(\\theta; X))$, the [[variance|0477R9PYAE]] of the score.\nIf moreover $\\log f$ is twice differentiable, then $I(\\theta) = -E_\\theta\\big[\\partial^2_\\theta \\log f(X; \\theta)\\big]$.\nFor $n$ independent observations with the same density, the information is $n I(\\theta)$.\n\nExample: let $X$ be [[normal|05AQE0QR0D]] with mean $\\theta$ and known variance $\\sigma^2$.\nThen $\\log f(x; \\theta) = -(x - \\theta)^2 / (2\\sigma^2) + c$, so $s = (X - \\theta)/\\sigma^2$.\nHence $I(\\theta) = E_\\theta[(X - \\theta)^2] / \\sigma^4 = 1/\\sigma^2$.\n\nNon-example of the regular behaviour: the uniform densities $f(x; \\theta) = 1/\\theta$ on $(0, \\theta)$.\nHere $s = -1/\\theta$ wherever it is defined, so $E_\\theta[s^2] = 1/\\theta^2$, but $E_\\theta[s] = -1/\\theta \\ne 0$.\nThe support depends on $\\theta$, differentiation under the integral fails, and the usual properties of $I(\\theta)$ do not hold.\n",
  "commit": "ff8a40749dc16545e3d2fc637f6b0196e81d0bfa",
  "front": {
    "defines": "G1Z3WSTE8Q",
    "kind": "definition",
    "msc": [
      "62B10",
      "62F10"
    ],
    "name": "Fisher information",
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/Fisher_information"
      }
    ],
    "tag": "HSTFKC0Y39",
    "uses": [
      "MSC0MR8VMH",
      "CP97GRVTQ0",
      "0WQNRCNY3M",
      "0477R9PYAE",
      "05AQE0QR0D"
    ]
  },
  "kind": "definition",
  "link": "https://tamnd.github.io/math2/node/HSTFKC0Y39/",
  "links": [
    {
      "field": "defines",
      "target": "G1Z3WSTE8Q"
    },
    {
      "field": "uses",
      "target": "MSC0MR8VMH"
    },
    {
      "field": "uses",
      "target": "CP97GRVTQ0"
    },
    {
      "field": "uses",
      "target": "0WQNRCNY3M"
    },
    {
      "field": "uses",
      "target": "0477R9PYAE"
    },
    {
      "field": "uses",
      "target": "05AQE0QR0D"
    }
  ],
  "markdown": "https://tamnd.github.io/math2/node/HSTFKC0Y39.md",
  "msc": [
    "62B10",
    "62F10"
  ],
  "name": "Fisher information",
  "prov": null,
  "role": "",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/HST/HSTFKC0Y39.md",
  "status": null,
  "tag": "HSTFKC0Y39",
  "uri": "m2://HSTFKC0Y39",
  "url": "https://tamnd.github.io/math2/wiki/fisher-information-definition/",
  "writing": {
    "tag": "HSTFKC0Y39",
    "kind": "definition",
    "level": "W0",
    "next": [
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      }
    ]
  }
}