{
  "aliases": null,
  "backlinks": null,
  "body": "\n## Definition\n\nLet $\\theta_1, \\dots, \\theta_N$ be i.i.d. with a [[prior|00NBBSXRQS]] distribution $G$ on a parameter space $\\Theta$, and given them let $X_1, \\dots, X_N$ be independent with $X_i \\sim f(\\cdot \\mid \\theta_i)$, for a known [[parametric model|Q0DXKAQXE1]] $f$.\nThe prior $G$ is unknown, either fully or as $G = G_\\eta$ with an unknown hyperparameter $\\eta$.\nAn *empirical Bayes method* proceeds in two steps.\n\n1. It estimates $G$ by some $\\hat{G}$ from $X_1, \\dots, X_N$, using that the $X_i$ are i.i.d. with the [[marginal|0AXP2Z5N9H]] density $m_G(x) = \\int_\\Theta f(x \\mid \\theta) \\, G(d\\theta)$.\n2. It makes inference on each $\\theta_i$ with the [[posterior|0KJ7ZS8AJA]] of $\\theta_i$ given $X_i$ computed under the prior $\\hat{G}$.\n\n## Remarks\n\nIn the parametric case a common choice is to take $\\hat{\\eta}$ maximising the marginal likelihood $\\prod_{i} m_{G_\\eta}(X_i)$.\nExample: let $X_i \\mid \\theta_i \\sim \\mathcal{N}(\\theta_i, 1)$ and $\\theta_i \\sim \\mathcal{N}(0, \\tau^2)$ with $\\tau$ unknown.\nThe posterior mean is $(1 - B) X_i$ with $B = 1/(1 + \\tau^2)$, and marginally $X_i \\sim \\mathcal{N}(0, 1 + \\tau^2)$.\nSince $\\sum_i X_i^2 / (1 + \\tau^2)$ has the chi-squared law with $N$ degrees of freedom, $(N - 2)/\\sum_i X_i^2$ is an unbiased estimator of $B$ for $N \\ge 3$; plugging it in gives the James–Stein estimator.\nExample: for Poisson data, $f(x \\mid \\theta) = e^{-\\theta}\\theta^x/x!$, the posterior mean is $E[\\theta \\mid X = x] = (x + 1)\\, m_G(x + 1)/m_G(x)$, and Robbins' method replaces $m_G$ by the observed frequencies of the counts.\nNon-example: a full Bayesian analysis puts a fixed hyperprior on $\\eta$ and integrates over it.\nThis is a hierarchical Bayes method, not an empirical Bayes method, since step 1 does not estimate $G$.\n",
  "commit": "ff8a40749dc16545e3d2fc637f6b0196e81d0bfa",
  "front": {
    "defines": "8GG2Z2E61R",
    "kind": "definition",
    "msc": [
      "62C12",
      "62F15"
    ],
    "name": "Empirical Bayes method",
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/Empirical_Bayes_method"
      }
    ],
    "tag": "56QC25W5AT",
    "uses": [
      "00NBBSXRQS",
      "0KJ7ZS8AJA",
      "0AXP2Z5N9H",
      "Q0DXKAQXE1"
    ]
  },
  "kind": "definition",
  "link": "https://tamnd.github.io/math2/node/56QC25W5AT/",
  "links": [
    {
      "field": "defines",
      "target": "8GG2Z2E61R"
    },
    {
      "field": "uses",
      "target": "00NBBSXRQS"
    },
    {
      "field": "uses",
      "target": "0KJ7ZS8AJA"
    },
    {
      "field": "uses",
      "target": "0AXP2Z5N9H"
    },
    {
      "field": "uses",
      "target": "Q0DXKAQXE1"
    }
  ],
  "markdown": "https://tamnd.github.io/math2/node/56QC25W5AT.md",
  "msc": [
    "62C12",
    "62F15"
  ],
  "name": "Empirical Bayes method",
  "prov": null,
  "role": "",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/56Q/56QC25W5AT.md",
  "status": null,
  "tag": "56QC25W5AT",
  "uri": "m2://56QC25W5AT",
  "url": "https://tamnd.github.io/math2/wiki/empirical-bayes-method-definition/",
  "writing": {
    "tag": "56QC25W5AT",
    "kind": "definition",
    "level": "W0",
    "next": [
      {
        "what": "an inventory row",
        "note": "the row of the inventory that the node accounts for",
        "have": 0,
        "want": 1
      },
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      }
    ]
  }
}