Conjugate prior

Remarks

Example: for $k$ successes in $n$ Bernoulli trials with chance $p \in (0, 1)$, the beta distributions form a conjugate family. Indeed, with prior density proportional to $p^{a - 1}(1 - p)^{b - 1}$, the posterior density is proportional to $p^{a + k - 1}(1 - p)^{b + n - k - 1}$, so the posterior is $\mathrm{Beta}(a + k, b + n - k)$. Non-example: for a Poisson observation $x$ with rate $\lambda > 0$, the log-normal family is not conjugate. The posterior density is proportional to $\lambda^{x} e^{-\lambda}$ times a log-normal density, and the factor $e^{-\lambda}$ cannot be written in log-normal form. The gamma family is conjugate for the Poisson rate: a $\mathrm{Gamma}(\alpha, \beta)$ prior, with rate $\beta$, gives the posterior $\mathrm{Gamma}(\alpha + x, \beta + 1)$. The family of all distributions on $\Theta$ is conjugate, because every posterior is a distribution on $\Theta$; so the notion is useful only for small families.