Math 2.0

Statistics Definition

Efficiency of an estimator

Builds on EstimatorBias of an estimatorVarianceFisher informationCramér–Rao boundSample mean

Definition

Let {f(⋅;θ):θ∈Θ}\set{f(\cdot; \theta) : \theta \in \Theta} be a parametric family of densities with Θ⊆ℝ\Theta \subseteq \R an open interval, regular enough for the Cramér–Rao bound to hold. Let X=(X1,…,Xn)X = (X_1, \dots, X_n) be the data, and let In(θ)I_n(\theta) be the Fisher information of XX, assumed finite and positive. Let TT be an unbiased estimator of θ\theta (its bias is 00) with finite positive variance.

  1. The efficiency of TT at θ\theta is e(T,θ)=1/In(θ)Var⁡θ(T)e(T, \theta) = \dfrac{1/I_n(\theta)}{\operatorname{Var}_\theta(T)}.
  2. TT is efficient if e(T,θ)=1e(T, \theta) = 1 for every θ∈Θ\theta \in \Theta.
  3. For two unbiased estimators T1,T2T_1, T_2 of θ\theta, the relative efficiency of T1T_1 with respect to T2T_2 is Var⁡θ(T2)/Var⁡θ(T1)\operatorname{Var}_\theta(T_2)/\operatorname{Var}_\theta(T_1).

Remarks

By the Cramér–Rao bound, 0<e(T,θ)≤10 < e(T, \theta) \le 1.

Example: for i.i.d. Xi∼𝒩(μ,σ2)X_i \sim \mathcal{N}(\mu, \sigma^2) with σ2\sigma^2 known, In(μ)=n/σ2I_n(\mu) = n/\sigma^2 and the sample mean has variance σ2/n\sigma^2/n, so e(Xˉ,μ)=1e(\bar{X}, \mu) = 1 and Xˉ\bar{X} is efficient.

Non-example: in the same model, T=X1T = X_1 is unbiased with variance σ2\sigma^2, so e(T,μ)=1/ne(T, \mu) = 1/n, and TT is not efficient when n≥2n \ge 2.

The asymptotic relative efficiency compares two estimators T1,n,T2,nT_{1,n}, T_{2,n} with n(Tj,n−θ)→d𝒩(0,vj(θ))\sqrt{n}(T_{j,n} - \theta) \xrightarrow{d} \mathcal{N}(0, v_j(\theta)) by the ratio v2(θ)/v1(θ)v_2(\theta)/v_1(\theta). For normal data, the asymptotic relative efficiency of the sample median with respect to the sample mean is 2/π≈0.642/\pi \approx 0.64.

Reading path

To understand this page from the ground up, read these 76 pages in order. The foundations come first.

  1. Bias of an estimator concept
  2. Construction of the reals from Cauchy sequences concept
  3. Element definition
  4. Estimator concept
  5. Fisher information concept
  6. Integers object
  7. Ordered field definition
  8. Proposition concept
  9. Random experiment concept
  10. Rational numbers object
  11. Real numbers object
  12. Score concept
  13. Statistic concept
  14. Variance concept
  15. Set definition
  16. Ring of integers statement
  17. Disjunction definition
  18. Implication definition
  19. Predicate definition
  20. Ordered pair definition
  21. Union definition
  22. Existential quantifier definition
  23. Sample space definition
  24. Universal quantifier definition
  25. The rationals form a field statement
  26. Cartesian product definition
  27. Subset definition
  28. Uniqueness quantifier definition
  29. The rationals form an ordered field statement
  30. Binary relation definition
  31. Universal set definition
  32. Power set definition
  33. Order on the reals definition
  34. Function definition
  35. Complement definition
  36. Interval definition
  37. Injective function definition
  38. Preimage of a set definition
  39. Sigma-algebra definition
  40. Countable set definition
  41. Event definition
  42. Borel sigma-algebra definition
  43. Probability axioms concept
  44. Probability measure definition
  45. Probability distribution definition
  46. Probability space definition
  47. Conditional probability definition
  48. Random variable definition
  49. Discrete random variable definition
  50. Law of a random variable definition
  51. Independence of two events definition
  52. Random variable with a density definition
  53. Joint probability distribution definition
  54. Identically distributed random variables definition
  55. Expected value definition
  56. Mutual independence of events definition
  57. Independence of two random variables definition
  58. Marginal distribution definition
  59. Variance definition
  60. Linearity of expectation statement
  61. Independent sigma-algebras definition
  62. Independence means the joint law is the product of the marginals statement
  63. Covariance definition
  64. Independence of random variables definition
  65. Variance of a sum statement
  66. Cauchy–Schwarz inequality for expectations statement
  67. Independent and identically distributed random variables definition
  68. Expectation of a product of independent random variables statement
  69. Statistic definition
  70. Independent random variables are uncorrelated statement
  71. Cramér–Rao bound statement
  72. Estimator definition
  73. Variance of a sum of independent random variables statement
  74. Bias of an estimator definition
  75. Sample mean definition
  76. Efficiency of an estimator definition

Further reading

  1. Web https://en.wikipedia.org/wiki/Efficiency_(statistics)
About this page
Tag1PKZW82H0C
URIm2://1PKZW82H0C
Kinddefinition
WritingW0 stub
MSC62F10, 62F12
DefinesEfficiency of an estimator
UsesEstimator, Bias of an estimator, Variance, Fisher information, Cramér–Rao bound, Sample mean

Markdown · JSON · Edit on GitHub

What this page needs for W1

  • an inventory row: the row of the inventory that the node accounts for (0 of 1)
  • a source with a locator (0 of 1)

The writing levels say how complete the text of a page is. They do not measure its truth.

What links here

No page links to this page yet.