Math 2.0

Statistics Definition

Kolmogorov–Smirnov test

Builds on Empirical distribution functionStatistical hypothesis testCumulative distribution functionIndependent and identically distributed random variables

Definition

Let X1,…,XnX_1, \dots, X_n be i.i.d. real random variables with cumulative distribution function FF, and let FnF_n be their empirical distribution function. Let F0F_0 be a fixed continuous cumulative distribution function. The one-sample Kolmogorov–Smirnov statistic is Dn=sup⁡x∈ℝ∣Fn(x)−F0(x)∣.D_n = \sup_{x \in \R} \abs{F_n(x) - F_0(x)}. The Kolmogorov–Smirnov test of the null hypothesis H0 ⁣:F=F0H_0 \colon F = F_0 at level α\alpha rejects H0H_0 when Dn>cn,αD_n > c_{n, \alpha}, where cn,αc_{n, \alpha} is chosen so that P(Dn>cn,α)≤αP(D_n > c_{n, \alpha}) \le \alpha under H0H_0.

Let two independent i.i.d. samples of sizes nn and mm have empirical distribution functions FnF_n and GmG_m. The two-sample statistic is Dn,m=sup⁡x∣Fn(x)−Gm(x)∣D_{n,m} = \sup_{x} \abs{F_n(x) - G_m(x)}. It tests whether the two samples have the same continuous distribution.

Remarks

Under H0H_0 the law of DnD_n is the same for every continuous F0F_0, because F0(Xi)F_0(X_i) is uniform on [0,1][0, 1]. As n→∞n \to \infty, nDn\sqrt{n} D_n converges in distribution to the Kolmogorov distribution, the law of sup⁡0≤t≤1∣B(t)∣\sup_{0 \le t \le 1} \abs{B(t)} for a Brownian bridge BB: lim⁡n→∞P(nDn≤t)=1−2∑k=1∞(−1)k−1e−2k2t2,t>0.\lim_{n \to \infty} P(\sqrt{n} D_n \le t) = 1 - 2 \sum_{k=1}^{\infty} (-1)^{k-1} e^{-2k^2 t^2}, \qquad t > 0. For ordered data x(1)≤⋯≤x(n)x_{(1)} \le \dots \le x_{(n)} the supremum is a finite maximum: Dn=max⁡imax⁡(i/n−F0(x(i)), F0(x(i))−(i−1)/n)D_n = \max_{i} \max\big(i/n - F_0(x_{(i)}), \, F_0(x_{(i)}) - (i - 1)/n\big).

Example: n=1n = 1, X1=0.3X_1 = 0.3 and F0(x)=xF_0(x) = x on [0,1][0, 1]. Then F1F_1 jumps from 00 to 11 at 0.30.3, and D1=max⁡(0.3,1−0.3)=0.7D_1 = \max(0.3, 1 - 0.3) = 0.7.

Non-example: testing normality with the mean and variance estimated from the same data is not this test. Then F0F_0 is not fixed in advance, and the null law of DnD_n changes; the corrected procedure is the Lilliefors test.

Reading path

To understand this page from the ground up, read these 52 pages in order. The foundations come first.

  1. Construction of the reals from Cauchy sequences concept
  2. Element definition
  3. Empirical distribution function concept
  4. Integers object
  5. Ordered field definition
  6. Proposition concept
  7. Random experiment concept
  8. Rational numbers object
  9. Real numbers object
  10. Set definition
  11. Ring of integers statement
  12. Disjunction definition
  13. Implication definition
  14. Predicate definition
  15. Ordered pair definition
  16. Union definition
  17. Sample space definition
  18. Existential quantifier definition
  19. The rationals form a field statement
  20. Universal quantifier definition
  21. Cartesian product definition
  22. Subset definition
  23. The rationals form an ordered field statement
  24. Uniqueness quantifier definition
  25. Binary relation definition
  26. Universal set definition
  27. Order on the reals definition
  28. Power set definition
  29. Complement definition
  30. Function definition
  31. Interval definition
  32. Preimage of a set definition
  33. Sigma-algebra definition
  34. Event definition
  35. Borel sigma-algebra definition
  36. Probability axioms concept
  37. Probability measure definition
  38. Probability distribution definition
  39. Probability space definition
  40. Conditional probability definition
  41. Random variable definition
  42. Independence of two events definition
  43. Cumulative distribution function definition
  44. Law of a random variable definition
  45. Mutual independence of events definition
  46. Identically distributed random variables definition
  47. Independent sigma-algebras definition
  48. Independence of random variables definition
  49. Independent and identically distributed random variables definition
  50. Statistic definition
  51. Statistical hypothesis test concept
  52. Kolmogorov–Smirnov test definition

Further reading

  1. Web https://en.wikipedia.org/wiki/Kolmogorov%E2%80%93Smirnov_test
About this page
TagSARNZPAKE5
URIm2://SARNZPAKE5
Kinddefinition
WritingW0 stub
MSC62G10, 62G30
DefinesKolmogorov–Smirnov test
UsesEmpirical distribution function, Statistical hypothesis test, Cumulative distribution function, Independent and identically distributed random variables

Markdown · JSON · Edit on GitHub

What this page needs for W1

  • an inventory row: the row of the inventory that the node accounts for (0 of 1)
  • a source with a locator (0 of 1)

The writing levels say how complete the text of a page is. They do not measure its truth.

What links here

No page links to this page yet.