Statistical decision theory

Remarks

A rule $\delta_1$ is at least as good as $\delta_2$ if $R(\theta, \delta_1) \le R(\theta, \delta_2)$ for all $\theta$. This is only a partial order, so the theory adds criteria: admissibility, the minimax principle and the Bayes principle. Randomized rules, which choose an action at random given the data, are often allowed; they make the set of risk functions convex.

Example: point estimation of a normal mean is the decision problem with $P_\theta$ the law of $n$ independent normal variables with mean $\theta$ and variance $1$. Here $\Theta = \mathcal{A} = \R$ and $L(\theta, a) = (a - \theta)^2$. The rule $\delta(X) = \bar X$ is a measurable map from $\R^n$ to $\mathcal{A}$, with risk $1/n$.

Non-example: a model and a set of estimators without a loss function do not form a decision problem. Clause 3 is missing, so risks are undefined and rules cannot be compared.