Math 2.0

Statistics Hypothesis testing Theorem

Wilks' theorem

Builds on Likelihood-ratio testChi-squared distributionConvergence in distributionIndependent and identically distributed random variables

Statement

Let X1,…,XnX_1, \dots, X_n be i.i.d. with density fθf_\theta, where θ\theta lies in an open set Θ⊆ℝk\Theta \subseteq \R^k. Assume the standard regularity conditions for the asymptotic normality of the maximum likelihood estimator, with a positive definite Fisher information. Let Θ0⊆Θ\Theta_0 \subseteq \Theta be the set of θ\theta whose first qq coordinates equal fixed values, with 1≤q≤k1 \le q \le k. Let λn\lambda_n be the likelihood-ratio statistic of H0 ⁣:θ∈Θ0H_0 \colon \theta \in \Theta_0 based on X1,…,XnX_1, \dots, X_n. If the true θ\theta lies in Θ0\Theta_0, then −2log⁡λn-2 \log \lambda_n converges in distribution to the chi-squared distribution with qq degrees of freedom as n→∞n \to \infty.

Remarks

The number qq is the difference between the dimension of Θ\Theta and the dimension of Θ0\Theta_0. Proof idea: expand the log-likelihood to second order around the unrestricted and the restricted maximum likelihood estimates. The difference becomes a quadratic form in an asymptotically normal vector, which has a limiting chi-squared law with qq degrees of freedom. Example: for an i.i.d. 𝒩(μ,σ2)\mathcal{N}(\mu, \sigma^2) sample and H0 ⁣:μ=0H_0 \colon \mu = 0, here k=2k = 2 and q=1q = 1, so one rejects when −2log⁡λn-2 \log \lambda_n exceeds the 1−α1 - \alpha quantile of χ12\chi^2_1. The hypothesis that the true parameter is an interior point matters. For a variance component tested at the boundary value 00, the limit law is a mixture of χ02\chi^2_0 and χ12\chi^2_1 rather than χ12\chi^2_1.

Going deeper

For a student

This is why likelihood-ratio tests are used everywhere: one computes −2log⁡λn-2 \log \lambda_n and compares it with a chi-squared quantile. The score test and the Wald test have the same limiting chi-squared law under the null hypothesis, and the three tests are asymptotically equivalent. The theorem needs regularity: smooth densities, identifiability, and a true parameter in the interior of the parameter set. It is an asymptotic statement, so for small samples the chi-squared approximation can be poor.

Reading path

To understand this page from the ground up, read these 80 pages in order. The foundations come first.

  1. Construction of the reals from Cauchy sequences concept
  2. Element definition
  3. Integers object
  4. Likelihood-ratio test concept
  5. Ordered field definition
  6. Proposition concept
  7. Random experiment concept
  8. Rational numbers object
  9. Real numbers object
  10. Set definition
  11. Ring of integers statement
  12. Disjunction definition
  13. Implication definition
  14. Predicate definition
  15. Ordered pair definition
  16. Union definition
  17. Sample space definition
  18. Existential quantifier definition
  19. The rationals form a field statement
  20. Universal quantifier definition
  21. Cartesian product definition
  22. Subset definition
  23. The rationals form an ordered field statement
  24. Uniqueness quantifier definition
  25. Binary relation definition
  26. Universal set definition
  27. Order on the reals definition
  28. Power set definition
  29. Complement definition
  30. Function definition
  31. Interval definition
  32. Indicator function definition
  33. Injective function definition
  34. Preimage of a set definition
  35. Sigma-algebra definition
  36. Countable set definition
  37. Event definition
  38. Borel sigma-algebra definition
  39. Probability axioms concept
  40. Probability measure definition
  41. Dirac measure definition
  42. Probability distribution definition
  43. Probability space definition
  44. Conditional probability definition
  45. Random variable definition
  46. Discrete random variable definition
  47. Cumulative distribution function definition
  48. Independence of two events definition
  49. Law of a random variable definition
  50. Random variable with a density definition
  51. Convergence in distribution definition
  52. Joint probability distribution definition
  53. Independence of two random variables definition
  54. Mutual independence of events definition
  55. Expected value definition
  56. Probability density function definition
  57. Identically distributed random variables definition
  58. Marginal distribution definition
  59. Variance definition
  60. Independent sigma-algebras definition
  61. Linearity of expectation statement
  62. Density of the normal distribution definition
  63. Exponential distribution definition
  64. Independence means the joint law is the product of the marginals statement
  65. Independence of random variables definition
  66. Covariance definition
  67. Normal distribution definition
  68. Gamma distribution definition
  69. Expectation of a product of independent random variables statement
  70. Variance of a sum statement
  71. Independent and identically distributed random variables definition
  72. Independent random variables are uncorrelated statement
  73. Statistic definition
  74. Variance of a sum of independent random variables statement
  75. Estimator definition
  76. Sample mean definition
  77. Bias of an estimator definition
  78. Sample variance definition
  79. Chi-squared distribution definition
  80. Wilks' theorem statement

Further reading

  1. Web https://en.wikipedia.org/wiki/Wilks%27_theorem
About this page
TagWQPB6GXD0M
URIm2://WQPB6GXD0M
Kindstatement, theorem
WritingW0 stub
MSC62F05, 62E20
StatusThis copy of the site has no build data. Run m2 status WQPB6GXD0M.
AboutLikelihood-ratio test
UsesLikelihood-ratio test, Chi-squared distribution, Convergence in distribution, Independent and identically distributed random variables
Same as

Markdown · JSON · Edit on GitHub

What this page needs for W1

  • an inventory row: the row of the inventory that the node accounts for (0 of 1)
  • a source with a locator (0 of 1)

The writing levels say how complete the text of a page is. They do not measure its truth.

What links here

No page links to this page yet.