Math 2.0

Statistics Point estimation Counterexample

Hodges' superefficient estimator

Builds on Hájek–Le Cam convolution theoremEfficiency of an estimatorSample meanNormal distributionIndependent and identically distributed random variablesConvergence in distributionFisher informationMean squared error

Counterexample

Let X1,…,XnX_1, \dots, X_n be i.i.d. 𝒩(θ,1)\mathcal{N}(\theta, 1), the normal distribution with unknown mean θ∈ℝ\theta \in \R, and let Xˉn\bar{X}_n be the sample mean. Hodges' estimator is Tn={Xˉnif ∣Xˉn∣≥n−1/4,0if ∣Xˉn∣<n−1/4.T_n = \begin{cases} \bar{X}_n & \text{if } \abs{\bar{X}_n} \ge n^{-1/4}, \\ 0 & \text{if } \abs{\bar{X}_n} < n^{-1/4}. \end{cases} The Fisher information of one observation is 11, and n(Xˉn−θ)∼𝒩(0,1)\sqrt{n}(\bar{X}_n - \theta) \sim \mathcal{N}(0, 1) exactly.

Check of the limit laws, with convergence in distribution:

  1. If θ≠0\theta \ne 0, then Pθ(∣Xˉn∣<n−1/4)→0P_\theta(\abs{\bar{X}_n} < n^{-1/4}) \to 0, because Xˉn→θ\bar{X}_n \to \theta while n−1/4→0n^{-1/4} \to 0. So Tn=XˉnT_n = \bar{X}_n with probability tending to one, and n(Tn−θ)→d𝒩(0,1)\sqrt{n}(T_n - \theta) \xrightarrow{d} \mathcal{N}(0, 1).
  2. If θ=0\theta = 0, then P0(∣Xˉn∣<n−1/4)=P(∣Z∣<n1/4)→1P_0(\abs{\bar{X}_n} < n^{-1/4}) = P(\abs{Z} < n^{1/4}) \to 1 for ZZ standard normal. So Tn=0T_n = 0 with probability tending to one, and n Tn→d0\sqrt{n}\, T_n \xrightarrow{d} 0.

So the asymptotic variance of TnT_n is 11 for θ≠0\theta \ne 0 and 00 for θ=0\theta = 0: at θ=0\theta = 0 it beats the bound 1/I(θ)=11/I(\theta) = 1 that maximum likelihood attains. This is superefficiency: the usual bound of efficiency is beaten at one point.

Remarks

The price appears near 00. Take θn=12n−1/4\theta_n = \frac{1}{2} n^{-1/4}. Then Xˉn−θn\bar{X}_n - \theta_n is of order n−1/2n^{-1/2}, so Pθn(Tn=0)→1P_{\theta_n}(T_n = 0) \to 1, and the scaled mean squared error satisfies n Eθn[(Tn−θn)2]≥nθn2Pθn(Tn=0)≈14n→∞n\, E_{\theta_n}[(T_n - \theta_n)^2] \ge n \theta_n^2 P_{\theta_n}(T_n = 0) \approx \frac{1}{4}\sqrt{n} \to \infty, while for Xˉn\bar{X}_n it equals 11.

TnT_n is not a regular estimator at θ=0\theta = 0: under θn=h/n\theta_n = h/\sqrt{n}, the law of n(Tn−θn)\sqrt{n}(T_n - \theta_n) tends to the point mass at −h-h, which depends on hh. This is why the Hájek–Le Cam convolution theorem assumes regularity.

Reading path

To understand this page from the ground up, read these 83 pages in order. The foundations come first.

  1. Construction of the reals from Cauchy sequences concept
  2. Efficiency of an estimator concept
  3. Element definition
  4. Fisher information concept
  5. Integers object
  6. Local asymptotic normality concept
  7. Ordered field definition
  8. Proposition concept
  9. Random experiment concept
  10. Rational numbers object
  11. Real numbers object
  12. Set definition
  13. Ring of integers statement
  14. Disjunction definition
  15. Implication definition
  16. Predicate definition
  17. Ordered pair definition
  18. Union definition
  19. Existential quantifier definition
  20. Sample space definition
  21. Universal quantifier definition
  22. The rationals form a field statement
  23. Cartesian product definition
  24. Subset definition
  25. Uniqueness quantifier definition
  26. The rationals form an ordered field statement
  27. Binary relation definition
  28. Universal set definition
  29. Power set definition
  30. Order on the reals definition
  31. Function definition
  32. Complement definition
  33. Interval definition
  34. Injective function definition
  35. Indicator function definition
  36. Preimage of a set definition
  37. Sigma-algebra definition
  38. Countable set definition
  39. Event definition
  40. Borel sigma-algebra definition
  41. Probability axioms concept
  42. Probability measure definition
  43. Probability distribution definition
  44. Dirac measure definition
  45. Probability space definition
  46. Conditional probability definition
  47. Random variable definition
  48. Cumulative distribution function definition
  49. Discrete random variable definition
  50. Law of a random variable definition
  51. Independence of two events definition
  52. Random variable with a density definition
  53. Convergence in distribution definition
  54. Joint probability distribution definition
  55. Probability mass function definition
  56. Expected value definition
  57. Identically distributed random variables definition
  58. Independence of two random variables definition
  59. Mutual independence of events definition
  60. Probability density function definition
  61. Marginal distribution definition
  62. Variance definition
  63. Linearity of expectation statement
  64. Convolution of probability distributions definition
  65. Independent sigma-algebras definition
  66. Density of the normal distribution definition
  67. Covariance definition
  68. Independence means the joint law is the product of the marginals statement
  69. Independence of random variables definition
  70. Normal distribution definition
  71. Variance of a sum statement
  72. Multivariate normal distribution definition
  73. Expectation of a product of independent random variables statement
  74. Independent and identically distributed random variables definition
  75. Statistic definition
  76. Independent random variables are uncorrelated statement
  77. Variance of a sum of independent random variables statement
  78. Estimator definition
  79. Sample mean definition
  80. Bias of an estimator definition
  81. Hájek–Le Cam convolution theorem statement
  82. Mean squared error definition
  83. Hodges' superefficient estimator counterexample

Further reading

  1. Web https://en.wikipedia.org/wiki/Hodges%27_estimator
About this page
Tag2E46CY6S3Z
URIm2://2E46CY6S3Z
Kindcounterexample
WritingW0 stub
MSC62F12
UsesHájek–Le Cam convolution theorem, Efficiency of an estimator, Sample mean, Normal distribution, Independent and identically distributed random variables, Convergence in distribution, Fisher information, Mean squared error
Same as

Markdown · JSON · Edit on GitHub

What this page needs for W1

  • an inventory row: the row of the inventory that the node accounts for (0 of 1)
  • a source with a locator (0 of 1)
  • edge refutes or not_instance_of out (0 of 1)

The writing levels say how complete the text of a page is. They do not measure its truth.

What links here

No page links to this page yet.