Math 2.0

Statistics Definition

Bayesian information criterion

Builds on Likelihood functionMaximum likelihood estimation

Definition

Let MM be a parametric statistical model with kk free real parameters, and let xx be data of nn observations. Let L^\hat L be the maximum of the likelihood function of MM at xx, the value at the maximum likelihood estimate. The Bayesian information criterion of MM at xx is BIC=kln⁡n−2ln⁡L^.\mathrm{BIC} = k \ln n - 2 \ln \hat L. Among several candidate models for the same data, the BIC rule selects a model with the smallest BIC.

Remarks

Under regularity conditions, −BIC/2-\mathrm{BIC}/2 approximates the logarithm of the marginal likelihood of MM, up to a term that stays bounded as n→∞n \to \infty. So differences of BIC approximate −2-2 times the logarithm of a Bayes factor. Example: with n=100n = 100, a model with k=2k = 2 and ln⁡L^=−100\ln \hat L = -100 has BIC=2ln⁡100+200≈209.2\mathrm{BIC} = 2 \ln 100 + 200 \approx 209.2. A model with k=3k = 3 and ln⁡L^=−98\ln \hat L = -98 has BIC≈209.8\mathrm{BIC} \approx 209.8, so the BIC rule keeps the smaller model, while the AIC values 204204 and 202202 favour the larger one. Non-example: 2k−2ln⁡L^2k - 2 \ln \hat L is the Akaike information criterion, not the BIC; the two agree only when ln⁡n=2\ln n = 2.

Reading path

To understand this page from the ground up, read these 74 pages in order. The foundations come first.

  1. Construction of the reals from Cauchy sequences concept
  2. Element definition
  3. Integers object
  4. Ordered field definition
  5. Proposition concept
  6. Random experiment concept
  7. Rational numbers object
  8. Real numbers object
  9. Set definition
  10. Ring of integers statement
  11. Disjunction definition
  12. Implication definition
  13. Predicate definition
  14. Ordered pair definition
  15. Union definition
  16. Sample space definition
  17. The rationals form a field statement
  18. Existential quantifier definition
  19. Universal quantifier definition
  20. Cartesian product definition
  21. Subset definition
  22. The rationals form an ordered field statement
  23. Uniqueness quantifier definition
  24. Binary relation definition
  25. Universal set definition
  26. Order on the reals definition
  27. Power set definition
  28. Complement definition
  29. Function definition
  30. Interval definition
  31. Injective function definition
  32. Preimage of a set definition
  33. Sigma-algebra definition
  34. Countable set definition
  35. Event definition
  36. Borel sigma-algebra definition
  37. Probability axioms concept
  38. Probability measure definition
  39. Probability distribution definition
  40. Probability space definition
  41. Conditional probability definition
  42. Random variable definition
  43. Cumulative distribution function definition
  44. Discrete random variable definition
  45. Law of a random variable definition
  46. Independence of two events definition
  47. Convergence in probability definition
  48. Random variable with a density definition
  49. Joint probability distribution definition
  50. Probability mass function definition
  51. Mutual independence of events definition
  52. Expected value definition
  53. Identically distributed random variables definition
  54. Independence of two random variables definition
  55. Probability density function definition
  56. Marginal distribution definition
  57. Independent sigma-algebras definition
  58. Variance definition
  59. Linearity of expectation statement
  60. Independence of random variables definition
  61. Independence means the joint law is the product of the marginals statement
  62. Covariance definition
  63. Independent and identically distributed random variables definition
  64. Variance of a sum statement
  65. Expectation of a product of independent random variables statement
  66. Statistic definition
  67. Likelihood function definition
  68. Independent random variables are uncorrelated statement
  69. Estimator definition
  70. Variance of a sum of independent random variables statement
  71. Sample mean definition
  72. Consistent estimator definition
  73. Maximum likelihood estimation concept
  74. Bayesian information criterion definition

Further reading

  1. Web https://en.wikipedia.org/wiki/Bayesian_information_criterion
About this page
TagH82J4YJF8N
URIm2://H82J4YJF8N
Kinddefinition
WritingW0 stub
MSC62F15, 62B10
DefinesBayesian information criterion
UsesLikelihood function, Maximum likelihood estimation

Markdown · JSON · Edit on GitHub

What this page needs for W1

  • an inventory row: the row of the inventory that the node accounts for (0 of 1)
  • a source with a locator (0 of 1)

The writing levels say how complete the text of a page is. They do not measure its truth.

What links here

No page links to this page yet.