Empirical Bayes method

Remarks

In the parametric case a common choice is to take $\hat{\eta}$ maximising the marginal likelihood $\prod_{i} m_{G_\eta}(X_i)$. Example: let $X_i \mid \theta_i \sim \mathcal{N}(\theta_i, 1)$ and $\theta_i \sim \mathcal{N}(0, \tau^2)$ with $\tau$ unknown. The posterior mean is $(1 - B) X_i$ with $B = 1/(1 + \tau^2)$, and marginally $X_i \sim \mathcal{N}(0, 1 + \tau^2)$. Since $\sum_i X_i^2 / (1 + \tau^2)$ has the chi-squared law with $N$ degrees of freedom, $(N - 2)/\sum_i X_i^2$ is an unbiased estimator of $B$ for $N \ge 3$; plugging it in gives the James–Stein estimator. Example: for Poisson data, $f(x \mid \theta) = e^{-\theta}\theta^x/x!$, the posterior mean is $E[\theta \mid X = x] = (x + 1)\, m_G(x + 1)/m_G(x)$, and Robbins' method replaces $m_G$ by the observed frequencies of the counts. Non-example: a full Bayesian analysis puts a fixed hyperprior on $\eta$ and integrates over it. This is a hierarchical Bayes method, not an empirical Bayes method, since step 1 does not estimate $G$.