Complete class theorem

Remarks

Consider the risk set $S = \set{(R(\theta_1, \delta), \dots, R(\theta_k, \delta))}$ in $\R^k$, over all randomized rules $\delta$. Mixing two rules mixes their risk points, so $S$ is a convex set. If $\delta$ is admissible, its risk point $x$ meets the convex set $\set{y : y_i \le x_i \text{ for all } i}$ in $S$ only at points equal to $x$. A separating hyperplane through $x$ then has a normal vector with nonnegative entries, and normalizing it gives the prior $\pi$. The minimized quantity is the Bayes risk of $\pi$.

A class $\mathcal{C}$ of rules is called complete if every rule outside $\mathcal{C}$ is dominated by some rule in $\mathcal{C}$. Complete class theorems are results of this kind, which show that the Bayes rules, or their limits, form a complete class. For infinite $\Theta$, similar results hold under extra conditions, with limits of Bayes rules or generalized Bayes rules in place of Bayes rules.

Randomization matters in the proof: without it, the risk set need not be convex.

Going deeper

For a student

The theorem gives a frequentist reason to use Bayesian methods: admissibility, a purely frequentist property, forces a rule to be Bayes. It reduces the search for admissible rules to the study of Bayes rules for all priors. The proof is a direct application of the separating hyperplane theorem to the convex risk set. In infinite-dimensional problems the analogues are more delicate, and admissible rules are typically limits of Bayes rules.