Math 2.0

Statistics Statistical decision theory Proposition

Bayes risk is at most minimax risk

Builds on Risk functionBayes riskMinimax estimator

Statement

Let {Pθ:θ∈Θ}\set{P_\theta : \theta \in \Theta} be a statistical model with a loss function, with risk function R(θ,δ)R(\theta, \delta). Let π\pi be a prior on Θ\Theta such that θ↦R(θ,δ)\theta \mapsto R(\theta, \delta) is measurable for every rule δ\delta. Then the Bayes risk of π\pi is at most the minimax risk: r(π)=inf⁡δ∫ΘR(θ,δ) π(dθ)≤inf⁡δsup⁡θ∈ΘR(θ,δ).r(\pi) = \inf_\delta \int_\Theta R(\theta, \delta) \, \pi(d\theta) \le \inf_\delta \sup_{\theta \in \Theta} R(\theta, \delta).

Remarks

For each fixed rule δ\delta, an average of R(⋅,δ)R(\cdot, \delta) is at most its supremum, so ∫R(θ,δ) π(dθ)≤sup⁡θR(θ,δ)\int R(\theta, \delta) \, \pi(d\theta) \le \sup_\theta R(\theta, \delta). Taking the infimum over δ\delta on both sides gives the claim. Hence also sup⁡πr(π)≤R∗\sup_\pi r(\pi) \le R^*, where R∗R^* is the minimax risk.

This inequality is the standard way to prove lower bounds for the minimax risk. If a rule δ∗\delta^* has sup⁡θR(θ,δ∗)=r(π)\sup_\theta R(\theta, \delta^*) = r(\pi) for some prior π\pi, then δ∗\delta^* is minimax. For instance, a Bayes rule with constant risk is minimax, as for the binomial proportion with the beta prior of parameters n/2\sqrt{n}/2 and n/2\sqrt{n}/2.

Going deeper

For a student

To show that no estimator can do better than some value in the worst case, it is enough to find a prior whose Bayes risk equals that value. This is the Bayesian route to minimax lower bounds, often used with a sequence of priors whose Bayes risks increase to the candidate value. For a finite parameter set with randomized rules allowed, minimax theorems show that the two sides are equal when the supremum over priors is taken.

Reading path

To understand this page from the ground up, read these 4 pages in order. The foundations come first.

Show the path as a graph
Each page uses the pages above it. The colour of the bar on a box shows how strong the evidence is, and the box also gives it as text. The list has the same pages.
  • no evidence level
  1. Bayes risk concept
  2. Minimax estimator concept
  3. Risk function concept
  4. Bayes risk is at most minimax risk statement

Further reading

  1. Web https://en.wikipedia.org/wiki/Minimax_estimator
About this page
TagFB692YN0BZ
URIm2://FB692YN0BZ
Kindstatement, proposition
WritingW0 stub
MSC62C20, 62C10
StatusThis copy of the site has no build data. Run m2 status FB692YN0BZ.
AboutBayes risk, Minimax estimator
UsesRisk function, Bayes risk, Minimax estimator

Markdown · JSON · Edit on GitHub

What this page needs for W1

  • an inventory row: the row of the inventory that the node accounts for (0 of 1)
  • a source with a locator (0 of 1)

The writing levels say how complete the text of a page is. They do not measure its truth.

What links here

No page links to this page yet.