Math 2.0

Statistics Definition

Estimating equations

Builds on EstimatorExpected valueIndependent and identically distributed random variablesScoreM-estimatorMethod of moments

Definition

Let {Pθ:θ∈Θ}\set{P_\theta : \theta \in \Theta} be a family of distributions on a set 𝒳\mathcal{X}, with Θ⊆ℝk\Theta \subseteq \R^k. An estimating function is a map g ⁣:𝒳×Θ→ℝkg \colon \mathcal{X} \times \Theta \to \R^k. It is unbiased if Eθ[g(X,θ)]=0E_\theta[g(X, \theta)] = 0 for every θ∈Θ\theta \in \Theta, where X∼PθX \sim P_\theta and EθE_\theta is the expectation. For i.i.d. observations X1,…,XnX_1, \dots, X_n, the estimating equation is ∑i=1ng(Xi,θ)=0,\sum_{i=1}^n g(X_i, \theta) = 0, and an estimator θ^n\hat{\theta}_n that solves it is an estimating-equation estimator.

Remarks

Example: the score g(x,θ)=∇θlog⁡f(x;θ)g(x, \theta) = \nabla_\theta \log f(x; \theta) is an unbiased estimating function under the usual regularity conditions, and its estimating equation gives the likelihood equations.

Example: in the method of moments for a Poisson mean, g(x,λ)=x−λg(x, \lambda) = x - \lambda has Eλ[X−λ]=0E_\lambda[X - \lambda] = 0, and the equation ∑i(Xi−λ)=0\sum_i (X_i - \lambda) = 0 gives λ^=Xˉ\hat{\lambda} = \bar{X}.

Non-example: for the Poisson mean, g(x,λ)=x−2λg(x, \lambda) = x - 2\lambda has Eλ[g(X,λ)]=−λ≠0E_\lambda[g(X, \lambda)] = -\lambda \ne 0, so it is not unbiased, and its root Xˉ/2\bar{X}/2 converges to λ/2\lambda/2, not to λ\lambda.

Estimators of this kind are the ψ\psi-type M-estimators. Under regularity conditions, n(θ^n−θ)→d𝒩(0,A−1BA−T)\sqrt{n}(\hat{\theta}_n - \theta) \xrightarrow{d} \mathcal{N}(0, A^{-1} B A^{-T}) with A=Eθ[∇θg(X,θ)]A = E_\theta[\nabla_\theta g(X, \theta)] and B=Eθ[g(X,θ)g(X,θ)T]B = E_\theta[g(X, \theta) g(X, \theta)^T].

Reading path

To understand this page from the ground up, read these 77 pages in order. The foundations come first.

  1. Construction of the reals from Cauchy sequences concept
  2. Element definition
  3. Integers object
  4. M-estimator concept
  5. Ordered field definition
  6. Proposition concept
  7. Random experiment concept
  8. Rational numbers object
  9. Real numbers object
  10. Score concept
  11. Set definition
  12. Ring of integers statement
  13. Disjunction definition
  14. Implication definition
  15. Predicate definition
  16. Ordered pair definition
  17. Union definition
  18. Sample space definition
  19. Existential quantifier definition
  20. The rationals form a field statement
  21. Universal quantifier definition
  22. Cartesian product definition
  23. Subset definition
  24. The rationals form an ordered field statement
  25. Uniqueness quantifier definition
  26. Binary relation definition
  27. Universal set definition
  28. Order on the reals definition
  29. Power set definition
  30. Complement definition
  31. Function definition
  32. Interval definition
  33. Injective function definition
  34. Preimage of a set definition
  35. Sigma-algebra definition
  36. Countable set definition
  37. Event definition
  38. Borel sigma-algebra definition
  39. Probability axioms concept
  40. Probability measure definition
  41. Probability distribution definition
  42. Probability space definition
  43. Conditional probability definition
  44. Random variable definition
  45. Cumulative distribution function definition
  46. Discrete random variable definition
  47. Law of a random variable definition
  48. Independence of two events definition
  49. Convergence in probability definition
  50. Random variable with a density definition
  51. Joint probability distribution definition
  52. Probability mass function definition
  53. Mutual independence of events definition
  54. Expected value definition
  55. Identically distributed random variables definition
  56. Independence of two random variables definition
  57. Probability density function definition
  58. Marginal distribution definition
  59. Independent sigma-algebras definition
  60. Variance definition
  61. Law of the unconscious statistician statement
  62. Linearity of expectation statement
  63. Independence of random variables definition
  64. Independence means the joint law is the product of the marginals statement
  65. Covariance definition
  66. Moment definition
  67. Independent and identically distributed random variables definition
  68. Variance of a sum statement
  69. Expectation of a product of independent random variables statement
  70. Statistic definition
  71. Independent random variables are uncorrelated statement
  72. Estimator definition
  73. Variance of a sum of independent random variables statement
  74. Sample mean definition
  75. Consistent estimator definition
  76. Method of moments concept
  77. Estimating equations definition

Further reading

  1. Web https://en.wikipedia.org/wiki/Estimating_equations
About this page
TagKV0F0E8YR2
URIm2://KV0F0E8YR2
Kinddefinition
WritingW0 stub
MSC62F10, 62F12
DefinesEstimating equations
UsesEstimator, Expected value, Independent and identically distributed random variables, Score, M-estimator, Method of moments

Markdown · JSON · Edit on GitHub

What this page needs for W1

  • an inventory row: the row of the inventory that the node accounts for (0 of 1)
  • a source with a locator (0 of 1)

The writing levels say how complete the text of a page is. They do not measure its truth.

What links here

No page links to this page yet.