Math 2.0

Statistics Bayesian inference Concept

Bayesian inference

Builds on Bayes' theoremPrior probabilityPosterior probabilityLikelihood functionBeta distribution

Summary

Bayesian inference is a method of statistical inference that treats an unknown parameter θ\theta as a random variable. It uses Bayes' theorem to update a probability distribution for θ\theta when data arrive. One starts from a prior distribution with density π(θ)\pi(\theta) and a model with likelihood L(θ∣x)=f(x∣θ)L(\theta \mid x) = f(x \mid \theta). The posterior distribution has the density π(θ∣x)=f(x∣θ) π(θ)∫Θf(x∣θ′) π(θ′) dθ′.\pi(\theta \mid x) = \frac{f(x \mid \theta)\, \pi(\theta)}{\int_\Theta f(x \mid \theta')\, \pi(\theta')\, d\theta'}.

Remarks

In short, the posterior is proportional to the likelihood times the prior. Example: with a Beta⁡(a,b)\operatorname{Beta}(a, b) prior for the success probability pp, and kk successes in nn Bernoulli(pp) trials, the posterior is Beta⁡(a+k,b+n−k)\operatorname{Beta}(a + k, b + n - k) (see beta distribution). The prior is conjugate. The posterior gives point estimates, such as the posterior mean, and credible intervals. For new observations, the posterior predictive distribution is f(x~∣x)=∫Θf(x~∣θ) π(θ∣x) dθf(\tilde{x} \mid x) = \int_\Theta f(\tilde{x} \mid \theta)\, \pi(\theta \mid x)\, d\theta, if x~\tilde{x} and xx are conditionally independent given θ\theta.

Reading path

To understand this page from the ground up, read these 69 pages in order. The foundations come first.

  1. Construction of the reals from Cauchy sequences concept
  2. Element definition
  3. Integers object
  4. Ordered field definition
  5. Proposition concept
  6. Random experiment concept
  7. Rational numbers object
  8. Real numbers object
  9. Set definition
  10. Ring of integers statement
  11. Conjunction definition
  12. Disjunction definition
  13. Implication definition
  14. Predicate definition
  15. Empty set object
  16. Family of sets definition
  17. Ordered pair definition
  18. Intersection definition
  19. Union definition
  20. Sample space definition
  21. The rationals form a field statement
  22. Existential quantifier definition
  23. Universal quantifier definition
  24. Disjoint sets definition
  25. Cartesian product definition
  26. Subset definition
  27. Union of a family of sets definition
  28. The rationals form an ordered field statement
  29. Uniqueness quantifier definition
  30. Universal set definition
  31. Binary relation definition
  32. Partition of a set definition
  33. Order on the reals definition
  34. Power set definition
  35. Complement definition
  36. Function definition
  37. Interval definition
  38. Injective function definition
  39. Preimage of a set definition
  40. Sigma-algebra definition
  41. Countable set definition
  42. Event definition
  43. Borel sigma-algebra definition
  44. Probability axioms concept
  45. Probability measure definition
  46. Probability distribution definition
  47. Probability space definition
  48. Conditional probability definition
  49. Random variable definition
  50. Cumulative distribution function definition
  51. Discrete random variable definition
  52. Law of a random variable definition
  53. Independence of two events definition
  54. Law of total probability statement
  55. Random variable with a density definition
  56. Probability mass function definition
  57. Mutual independence of events definition
  58. Bayes' theorem statement
  59. Identically distributed random variables definition
  60. Probability density function definition
  61. Independent sigma-algebras definition
  62. Prior probability concept
  63. Continuous uniform distribution definition
  64. Independence of random variables definition
  65. Posterior probability concept
  66. Beta distribution definition
  67. Independent and identically distributed random variables definition
  68. Likelihood function definition
  69. Bayesian inference concept

Where this leads

1 page builds on this one.

Further reading

  1. Web https://en.wikipedia.org/wiki/Bayesian_inference, Formal description
  2. Web https://en.wikipedia.org/wiki/Bayesian_inference, Conjugate priors
About this page
Tag0JWRJX2RMP
URIm2://0JWRJX2RMP
Aliasesbayesian-statistics, bayesian-updating
Kindconcept
WritingW0 stub
MSC62F15
UsesBayes' theorem, Prior probability, Posterior probability, Likelihood function, Beta distribution
Same as

Markdown · JSON · Edit on GitHub

What this page needs for W1

  • an inventory row: the row of the inventory that the node accounts for (0 of 1)

The writing levels say how complete the text of a page is. They do not measure its truth.

What links here

Used by