---
tag: 1E8QNK0Q92
kind: statement
role: theorem
name: Bernstein–von Mises theorem
msc: [62F15, 62F12]
uses: [Q0DXKAQXE1, 0KJ7ZS8AJA, 00NBBSXRQS, G1Z3WSTE8Q, 0G558GJQMW, 3SZESKX3N5, 0EFDDXTVD6, 0AJN7WHBJZ]
about: [0KJ7ZS8AJA]
sources:
  - url: https://en.wikipedia.org/wiki/Bernstein%E2%80%93von_Mises_theorem
same_as:
  wikidata: Q4894580
  wikipedia: Bernstein–von Mises theorem
---

## Statement

Let $\set{P_\theta : \theta \in \Theta}$ be a [[parametric model|Q0DXKAQXE1]] with $\Theta \subseteq \R^k$ open, and let $X_1, X_2, \dots$ be [[i.i.d.|0AJN7WHBJZ]] with law $P_{\theta_0}$ for some $\theta_0 \in \Theta$.
Assume the following.

1. The model is differentiable in quadratic mean at $\theta_0$, with nonsingular [[Fisher information|G1Z3WSTE8Q]] matrix $I(\theta_0)$.
2. For every $\varepsilon > 0$ there are tests $\varphi_n = \varphi_n(X_1, \dots, X_n)$ with $E_{\theta_0}[\varphi_n] \to 0$ and $\sup_{\norm{\theta - \theta_0} \ge \varepsilon} E_\theta[1 - \varphi_n] \to 0$.
3. The [[prior|00NBBSXRQS]] has a density with respect to Lebesgue measure that is continuous and positive in a neighbourhood of $\theta_0$.

Let $\hat{\theta}_n$ be an efficient estimator, that is, $\sqrt{n}(\hat{\theta}_n - \theta_0) = \frac{1}{\sqrt{n}} \sum_{i=1}^{n} I(\theta_0)^{-1} \dot{\ell}_{\theta_0}(X_i) + o_P(1)$, where $\dot{\ell}_{\theta_0}$ is the score function.
Then the [[posterior distribution|0KJ7ZS8AJA]] $\Pi_n$ of $\theta$ given $X_1, \dots, X_n$ satisfies
$$\norm{\Pi_n - \mathcal{N}\left(\hat{\theta}_n, \tfrac{1}{n} I(\theta_0)^{-1}\right)}_{TV} \to 0$$
in $P_{\theta_0}$-[[probability|0EFDDXTVD6]], where $\norm{\cdot}_{TV}$ is the [[total variation distance|3SZESKX3N5]].

## Remarks

The proof expands the log-likelihood near $\theta_0$: by local asymptotic normality it is, up to $o_P(1)$, a quadratic function of $\sqrt{n}(\theta - \theta_0)$.
The tests of hypothesis 2 show that the posterior mass outside shrinking neighbourhoods of $\theta_0$ vanishes, and hypothesis 3 makes the prior nearly constant on these neighbourhoods.
So the posterior is close to the normalised exponential of a quadratic form, that is, to a [[multivariate normal|0G558GJQMW]] law.
Under the usual regularity conditions the maximum likelihood estimator is efficient and can serve as $\hat{\theta}_n$.
Example: for i.i.d. Bernoulli($p_0$) data with $0 < p_0 < 1$ and any beta prior, the posterior of $p$ is approximately $\mathcal{N}(\hat{p}_n, \hat{p}_n(1 - \hat{p}_n)/n)$ for large $n$, with $\hat{p}_n$ the proportion of successes.
The prior must charge every neighbourhood of $\theta_0$: a prior that vanishes near $\theta_0$ gives a posterior that stays away from $\theta_0$, and the conclusion fails.
The theorem can fail in nonparametric models.

## Explanations

### L1 human

With enough data, the prior stops mattering much.
The posterior then looks like a bell curve centred at the best classical estimate, with the spread that classical theory predicts.
So, in large samples, Bayesian credible intervals and frequentist confidence intervals nearly agree.

### L2 human

The theorem is the bridge between [[Bayesian inference|0JWRJX2RMP]] and frequentist asymptotics in regular parametric models.
It implies that an equal-tailed $95\%$ credible interval for a coordinate of $\theta$ has frequentist coverage tending to $95\%$.
Under additional integrability conditions, the posterior mean is also asymptotically efficient, like the maximum likelihood estimator.
Its hypotheses are the same local asymptotic normality conditions that drive the asymptotic theory of the maximum likelihood estimator.