Bayesian hierarchical model

Remarks

Models with more levels add further hyperpriors in the same way. Example: data in $J$ groups with $X_{ij} \mid \theta_j \sim \mathcal{N}(\theta_j, \sigma^2)$, group means $\theta_j \mid \mu, \tau \sim \mathcal{N}(\mu, \tau^2)$ i.i.d., and a hyperprior on $(\mu, \tau)$. All three clauses hold, with $\theta = (\theta_1, \dots, \theta_J)$ and $\varphi = (\mu, \tau)$. Given $\mu$ and $\tau$, the posterior mean of each $\theta_j$ lies between the sample mean of group $j$ and the overall level $\mu$. Non-example: the same model with $\mu$ and $\tau$ fixed numbers has no hyperprior, so clause 3 fails; it is an ordinary one-level Bayesian model. Integrating out $\theta$ gives the marginal $\int f(x \mid \theta)\pi(\theta \mid \varphi)\,d\theta$; estimating $\varphi$ from it instead of using a hyperprior is the empirical Bayes approach.