Math 2.0

Statistics Definition

Empirical likelihood

Builds on Independent and identically distributed random variablesLikelihood functionConvex hullChi-squared distribution

Definition

Let X1,…,XnX_1, \dots, X_n be i.i.d. random vectors in ℝd\R^d, and let μ∈ℝd\mu \in \R^d. The empirical likelihood ratio for the mean μ\mu is ℛ(μ)=max⁡{∏i=1nnwi  :  wi≥0, ∑i=1nwi=1, ∑i=1nwiXi=μ},\mathcal{R}(\mu) = \max\Big\{ \prod_{i=1}^{n} n w_i \;:\; w_i \ge 0, \ \sum_{i=1}^{n} w_i = 1, \ \sum_{i=1}^{n} w_i X_i = \mu \Big\}, with ℛ(μ)=0\mathcal{R}(\mu) = 0 when no weights satisfy the constraints. More generally, for a parameter θ\theta defined by E[m(X,θ)]=0E[m(X, \theta)] = 0, the constraint ∑iwiXi=μ\sum_i w_i X_i = \mu is replaced by ∑iwim(Xi,θ)=0\sum_i w_i m(X_i, \theta) = 0.

Remarks

The factor nn makes the unconstrained maximum equal to 11: without the mean constraint, ∏wi\prod w_i is largest at wi=1/nw_i = 1/n. Feasible weights exist exactly when μ\mu lies in the convex hull of X1,…,XnX_1, \dots, X_n. When μ\mu is in the interior of the hull, the maximiser is wi=1n(1+λT(Xi−μ))w_i = \frac{1}{n(1 + \lambda^{T}(X_i - \mu))}, where λ∈ℝd\lambda \in \R^d solves ∑iXi−μ1+λT(Xi−μ)=0\sum_i \frac{X_i - \mu}{1 + \lambda^{T}(X_i - \mu)} = 0. Owen's theorem: if μ0=E[X1]\mu_0 = E[X_1] and the covariance matrix of X1X_1 is finite and nonsingular, then −2ln⁡ℛ(μ0)-2 \ln \mathcal{R}(\mu_0) converges in distribution to the chi-squared distribution with dd degrees of freedom. So {μ:−2ln⁡ℛ(μ)≤c}\set{\mu : -2\ln\mathcal{R}(\mu) \le c}, with cc the (1−α)(1-\alpha)-quantile of χd2\chi^2_d, is an asymptotic confidence region of level 1−α1 - \alpha.

Example: μ=Xˉ\mu = \bar X. The weights wi=1/nw_i = 1/n satisfy the constraint, so ℛ(Xˉ)=1\mathcal{R}(\bar X) = 1, the largest possible value.

Non-example: data (1,2,3)(1, 2, 3) in ℝ\R and μ=5\mu = 5. No weighted average of 1,2,31, 2, 3 equals 55, because 55 is outside the convex hull [1,3][1, 3], so ℛ(5)=0\mathcal{R}(5) = 0.

Reading path

To understand this page from the ground up, read these 83 pages in order. The foundations come first.

  1. Affine combination concept
  2. Construction of the reals from Cauchy sequences concept
  3. Convex set concept
  4. Element definition
  5. Integers object
  6. Ordered field definition
  7. Proposition concept
  8. Random experiment concept
  9. Rational numbers object
  10. Real numbers object
  11. Convex hull definition
  12. Set definition
  13. Ring of integers statement
  14. Disjunction definition
  15. Implication definition
  16. Predicate definition
  17. Ordered pair definition
  18. Union definition
  19. Sample space definition
  20. Existential quantifier definition
  21. The rationals form a field statement
  22. Universal quantifier definition
  23. Cartesian product definition
  24. Subset definition
  25. Uniqueness quantifier definition
  26. The rationals form an ordered field statement
  27. Binary relation definition
  28. Power set definition
  29. Universal set definition
  30. Order on the reals definition
  31. Function definition
  32. Complement definition
  33. Interval definition
  34. Injective function definition
  35. Indicator function definition
  36. Preimage of a set definition
  37. Sigma-algebra definition
  38. Countable set definition
  39. Event definition
  40. Borel sigma-algebra definition
  41. Probability axioms concept
  42. Probability measure definition
  43. Probability distribution definition
  44. Dirac measure definition
  45. Probability space definition
  46. Conditional probability definition
  47. Random variable definition
  48. Cumulative distribution function definition
  49. Discrete random variable definition
  50. Law of a random variable definition
  51. Independence of two events definition
  52. Random variable with a density definition
  53. Joint probability distribution definition
  54. Probability mass function definition
  55. Expected value definition
  56. Identically distributed random variables definition
  57. Independence of two random variables definition
  58. Mutual independence of events definition
  59. Probability density function definition
  60. Marginal distribution definition
  61. Variance definition
  62. Linearity of expectation statement
  63. Independent sigma-algebras definition
  64. Density of the normal distribution definition
  65. Exponential distribution definition
  66. Covariance definition
  67. Independence means the joint law is the product of the marginals statement
  68. Independence of random variables definition
  69. Normal distribution definition
  70. Gamma distribution definition
  71. Variance of a sum statement
  72. Expectation of a product of independent random variables statement
  73. Independent and identically distributed random variables definition
  74. Statistic definition
  75. Independent random variables are uncorrelated statement
  76. Likelihood function definition
  77. Estimator definition
  78. Variance of a sum of independent random variables statement
  79. Bias of an estimator definition
  80. Sample mean definition
  81. Sample variance definition
  82. Chi-squared distribution definition
  83. Empirical likelihood definition

Further reading

  1. Web https://en.wikipedia.org/wiki/Empirical_likelihood
About this page
TagH4DD38RKBA
URIm2://H4DD38RKBA
Kinddefinition
WritingW0 stub
MSC62G10, 62G15, 62G20
DefinesEmpirical likelihood
UsesIndependent and identically distributed random variables, Likelihood function, Convex hull, Chi-squared distribution

Markdown · JSON · Edit on GitHub

What this page needs for W1

  • an inventory row: the row of the inventory that the node accounts for (0 of 1)
  • a source with a locator (0 of 1)

The writing levels say how complete the text of a page is. They do not measure its truth.

What links here

No page links to this page yet.