Math 2.0

Statistics Point estimation Concept

Maximum likelihood estimation

Builds on Likelihood functionEstimatorConsistent estimatorSample mean

Summary

Maximum likelihood estimation is a method to build estimators of unknown parameters. For observed data xx, the maximum likelihood estimate is a value θ^\hat{\theta} that maximizes the likelihood function θ↦L(θ∣x)\theta \mapsto L(\theta \mid x) over the parameter set Θ\Theta. In other words, it is the parameter value under which the observed data are most probable.

Remarks

The maximizer of LL is also the maximizer of the log-likelihood ℓ=ln⁡L\ell = \ln L. For a smooth ℓ\ell and an interior maximum, θ^\hat{\theta} solves the likelihood equations ∂ℓ/∂θj=0\partial \ell / \partial \theta_j = 0. Example: for kk successes in nn Bernoulli(pp) trials, ℓ(p)=kln⁡p+(n−k)ln⁡(1−p)\ell(p) = k \ln p + (n - k) \ln(1 - p), so p^=k/n\hat{p} = k/n. Example: for an i.i.d. 𝒩(μ,σ2)\mathcal{N}(\mu, \sigma^2) sample, μ^=xˉ\hat{\mu} = \bar{x} and σ^2=1n∑i(xi−xˉ)2\hat{\sigma}^2 = \frac{1}{n}\sum_i (x_i - \bar{x})^2. This estimator of σ2\sigma^2 is biased. Under regularity conditions, the estimator is consistent and asymptotically normal. Also, for a function gg, the maximum likelihood estimate of g(θ)g(\theta) is g(θ^)g(\hat{\theta}).

Reading path

To understand this page from the ground up, read these 73 pages in order. The foundations come first.

  1. Construction of the reals from Cauchy sequences concept
  2. Element definition
  3. Integers object
  4. Ordered field definition
  5. Proposition concept
  6. Random experiment concept
  7. Rational numbers object
  8. Real numbers object
  9. Set definition
  10. Ring of integers statement
  11. Disjunction definition
  12. Implication definition
  13. Predicate definition
  14. Ordered pair definition
  15. Union definition
  16. Sample space definition
  17. The rationals form a field statement
  18. Existential quantifier definition
  19. Universal quantifier definition
  20. Cartesian product definition
  21. Subset definition
  22. The rationals form an ordered field statement
  23. Uniqueness quantifier definition
  24. Binary relation definition
  25. Universal set definition
  26. Order on the reals definition
  27. Power set definition
  28. Complement definition
  29. Function definition
  30. Interval definition
  31. Injective function definition
  32. Preimage of a set definition
  33. Sigma-algebra definition
  34. Countable set definition
  35. Event definition
  36. Borel sigma-algebra definition
  37. Probability axioms concept
  38. Probability measure definition
  39. Probability distribution definition
  40. Probability space definition
  41. Conditional probability definition
  42. Random variable definition
  43. Cumulative distribution function definition
  44. Discrete random variable definition
  45. Law of a random variable definition
  46. Independence of two events definition
  47. Convergence in probability definition
  48. Random variable with a density definition
  49. Joint probability distribution definition
  50. Probability mass function definition
  51. Mutual independence of events definition
  52. Expected value definition
  53. Identically distributed random variables definition
  54. Independence of two random variables definition
  55. Probability density function definition
  56. Marginal distribution definition
  57. Independent sigma-algebras definition
  58. Variance definition
  59. Linearity of expectation statement
  60. Independence of random variables definition
  61. Independence means the joint law is the product of the marginals statement
  62. Covariance definition
  63. Independent and identically distributed random variables definition
  64. Variance of a sum statement
  65. Expectation of a product of independent random variables statement
  66. Statistic definition
  67. Likelihood function definition
  68. Independent random variables are uncorrelated statement
  69. Estimator definition
  70. Variance of a sum of independent random variables statement
  71. Sample mean definition
  72. Consistent estimator definition
  73. Maximum likelihood estimation concept

Where this leads

15 pages build on this one.

Further reading

  1. Web https://en.wikipedia.org/wiki/Maximum_likelihood_estimation, Principles
  2. Web https://en.wikipedia.org/wiki/Maximum_likelihood_estimation, Properties
About this page
Tag0FREPFKNS7
URIm2://0FREPFKNS7
Aliasesmle, maximum-likelihood-estimator, maximum-likelihood
Kindconcept
WritingW1 stated
MSC62F10
UsesLikelihood function, Estimator, Consistent estimator, Sample mean
Same as

Markdown · JSON · Edit on GitHub

What this page needs for W2

  • edge defines in: at least one definition (0 of 1)
  • edge instance_of in: at least two examples (0 of 2)
  • edge not_instance_of in: at least one non-example (0 of 1)
  • edge specializes both: the more general notions and the special cases (0 of 1)

The writing levels say how complete the text of a page is. They do not measure its truth.

What links here

Used by

Results about this