{
  "aliases": null,
  "backlinks": null,
  "body": "\n## Definition\n\nLet $\\set{f(\\cdot\\,; \\theta) : \\theta \\in \\Theta}$ be a family of densities with $\\Theta \\subseteq \\R^d$ open.\nLet $x_1, \\dots, x_n$ be observed data, and let\n$$\\ell(\\theta) = \\sum_{i=1}^{n} \\log f(x_i; \\theta)$$\nbe the log-[[likelihood|0VW9E4QXR6]], assumed twice differentiable in $\\theta$.\nThe *observed information* at $\\theta$ is the $d \\times d$ matrix\n$$J(\\theta) = -\\nabla^2 \\ell(\\theta), \\qquad J_{jk}(\\theta) = -\\frac{\\partial^2 \\ell(\\theta)}{\\partial \\theta_j \\, \\partial \\theta_k},$$\nthat is, minus the [[Hessian matrix|0693Y2METT]] of the log-likelihood.\nFor $d = 1$ it is the number $J(\\theta) = -\\ell''(\\theta)$.\n\n## Remarks\n\nIt is usually evaluated at the [[maximum likelihood estimate|0FREPFKNS7]] $\\hat\\theta$, and $J(\\hat\\theta)$ is then positive semidefinite when $\\hat\\theta$ is an interior maximum.\nIf the data are independent with density $f(\\cdot\\,; \\theta)$, then under regularity conditions $E_\\theta[J(\\theta)] = n I(\\theta)$, where $I$ is the [[Fisher information|G1Z3WSTE8Q]].\nThe inverse $J(\\hat\\theta)^{-1}$ is a standard estimate of the covariance matrix of $\\hat\\theta$.\n\nExample: [[Poisson|0XB9GQTN9G]] data with mean $\\lambda \u003e 0$ and $t = x_1 + \\cdots + x_n \u003e 0$.\nThen $\\ell(\\lambda) = t \\log \\lambda - n\\lambda - \\sum_i \\log(x_i!)$, so $J(\\lambda) = t/\\lambda^2$.\nAt $\\hat\\lambda = t/n$ this gives $J(\\hat\\lambda) = n^2/t$.\n\nNon-example: minus the second derivative of the likelihood $L = e^{\\ell}$ itself is not the observed information.\nIn the Poisson example with $n = 1$ and $t = 1$, $-L''(\\lambda)$ at $\\hat\\lambda = 1$ equals $e^{-1}$, while $J(1) = 1$.\n",
  "commit": "ff8a40749dc16545e3d2fc637f6b0196e81d0bfa",
  "front": {
    "defines": "KJ9ZQWYBY6",
    "kind": "definition",
    "msc": [
      "62F10",
      "62B10"
    ],
    "name": "Observed information",
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/Observed_information"
      }
    ],
    "tag": "ZPPN50CEEX",
    "uses": [
      "0VW9E4QXR6",
      "0693Y2METT",
      "0FREPFKNS7",
      "G1Z3WSTE8Q",
      "0XB9GQTN9G"
    ]
  },
  "kind": "definition",
  "link": "https://tamnd.github.io/math2/node/ZPPN50CEEX/",
  "links": [
    {
      "field": "defines",
      "target": "KJ9ZQWYBY6"
    },
    {
      "field": "uses",
      "target": "0VW9E4QXR6"
    },
    {
      "field": "uses",
      "target": "0693Y2METT"
    },
    {
      "field": "uses",
      "target": "0FREPFKNS7"
    },
    {
      "field": "uses",
      "target": "G1Z3WSTE8Q"
    },
    {
      "field": "uses",
      "target": "0XB9GQTN9G"
    }
  ],
  "markdown": "https://tamnd.github.io/math2/node/ZPPN50CEEX.md",
  "msc": [
    "62F10",
    "62B10"
  ],
  "name": "Observed information",
  "prov": null,
  "role": "",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/ZPP/ZPPN50CEEX.md",
  "status": null,
  "tag": "ZPPN50CEEX",
  "uri": "m2://ZPPN50CEEX",
  "url": "https://tamnd.github.io/math2/wiki/observed-information-definition/",
  "writing": {
    "tag": "ZPPN50CEEX",
    "kind": "definition",
    "level": "W0",
    "next": [
      {
        "what": "an inventory row",
        "note": "the row of the inventory that the node accounts for",
        "have": 0,
        "want": 1
      },
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      }
    ]
  }
}