---
tag: 56QC25W5AT
kind: definition
name: Empirical Bayes method
msc: [62C12, 62F15]
defines: 8GG2Z2E61R
uses: [00NBBSXRQS, 0KJ7ZS8AJA, 0AXP2Z5N9H, Q0DXKAQXE1]
sources:
  - url: https://en.wikipedia.org/wiki/Empirical_Bayes_method
---

## Definition

Let $\theta_1, \dots, \theta_N$ be i.i.d. with a [[prior|00NBBSXRQS]] distribution $G$ on a parameter space $\Theta$, and given them let $X_1, \dots, X_N$ be independent with $X_i \sim f(\cdot \mid \theta_i)$, for a known [[parametric model|Q0DXKAQXE1]] $f$.
The prior $G$ is unknown, either fully or as $G = G_\eta$ with an unknown hyperparameter $\eta$.
An *empirical Bayes method* proceeds in two steps.

1. It estimates $G$ by some $\hat{G}$ from $X_1, \dots, X_N$, using that the $X_i$ are i.i.d. with the [[marginal|0AXP2Z5N9H]] density $m_G(x) = \int_\Theta f(x \mid \theta) \, G(d\theta)$.
2. It makes inference on each $\theta_i$ with the [[posterior|0KJ7ZS8AJA]] of $\theta_i$ given $X_i$ computed under the prior $\hat{G}$.

## Remarks

In the parametric case a common choice is to take $\hat{\eta}$ maximising the marginal likelihood $\prod_{i} m_{G_\eta}(X_i)$.
Example: let $X_i \mid \theta_i \sim \mathcal{N}(\theta_i, 1)$ and $\theta_i \sim \mathcal{N}(0, \tau^2)$ with $\tau$ unknown.
The posterior mean is $(1 - B) X_i$ with $B = 1/(1 + \tau^2)$, and marginally $X_i \sim \mathcal{N}(0, 1 + \tau^2)$.
Since $\sum_i X_i^2 / (1 + \tau^2)$ has the chi-squared law with $N$ degrees of freedom, $(N - 2)/\sum_i X_i^2$ is an unbiased estimator of $B$ for $N \ge 3$; plugging it in gives the James–Stein estimator.
Example: for Poisson data, $f(x \mid \theta) = e^{-\theta}\theta^x/x!$, the posterior mean is $E[\theta \mid X = x] = (x + 1)\, m_G(x + 1)/m_G(x)$, and Robbins' method replaces $m_G$ by the observed frequencies of the counts.
Non-example: a full Bayesian analysis puts a fixed hyperprior on $\eta$ and integrates over it.
This is a hierarchical Bayes method, not an empirical Bayes method, since step 1 does not estimate $G$.