{
  "aliases": null,
  "backlinks": null,
  "body": "\n## Statement\n\nLet $X_1, \\dots, X_n$ be [[i.i.d.|0AJN7WHBJZ]] real random variables with [[cumulative distribution function|0MXMS3AE13]] $F$.\nLet $F_n(x) = \\frac{1}{n} \\#\\set{i : X_i \\le x}$ be their [[empirical distribution function|A05WR6TEK5]].\nThen for every $\\varepsilon \u003e 0$,\n$$P\\Big( \\sup_{x \\in \\R} \\abs{F_n(x) - F(x)} \u003e \\varepsilon \\Big) \\le 2 e^{-2 n \\varepsilon^2}.$$\n\n## Remarks\n\nDvoretzky, Kiefer and Wolfowitz proved in 1956 a bound of the form $C e^{-2n\\varepsilon^2}$ with an unspecified constant $C$.\nMassart showed in 1990 that $C = 2$ works, and this constant cannot be lowered.\nThe reduction to the uniform law on $[0, 1]$ uses the quantile transform.\nThe variable $X_i$ has the same law as $Q(U_i)$, for $U_i$ uniform and $Q$ the quantile function of $F$.\nThe supremum can only shrink under this change.\nThe hard part is the sharp tail bound for the uniform case.\n\nExample: a confidence band.\nGiven $\\alpha \\in (0, 1)$, put $\\varepsilon = \\sqrt{\\ln(2/\\alpha) / (2n)}$, so that $2e^{-2n\\varepsilon^2} = \\alpha$.\nThen the band $F_n(x) \\pm \\varepsilon$ contains the whole graph of $F$ with probability at least $1 - \\alpha$.\nFor $n = 1000$ and $\\alpha = 0.05$, this gives $\\varepsilon \\approx 0.043$.\n\nThe bound needs no assumption on $F$: no continuity and no moments.\nIt does need independence and a common law; for dependent data $F_n$ can stay far from $F$.\n\n## Explanations\n\n### L1 human\n\nThe empirical distribution function of a sample is the natural guess for the true distribution function.\nThis inequality says how good the guess is, uniformly over all $x$ at once.\nThe chance that the guess is off by more than $\\varepsilon$ anywhere decays like $e^{-2n\\varepsilon^2}$, whatever the distribution is.\nSo a sample of a few thousand points pins down the whole distribution function to within a few hundredths.\n\n### L2 human\n\nThe inequality is a quantitative form of the [[Glivenko–Cantelli theorem|E7B78X780X]].\nSince $\\sum_n 2e^{-2n\\varepsilon^2} \u003c \\infty$, the Borel–Cantelli lemma gives $\\sup_x \\abs{F_n - F} \\to 0$ almost surely.\nIt also bounds the tail of the [[Kolmogorov–Smirnov|YZYHWC6C5F]] statistic: $P(\\sqrt{n} D_n \u003e t) \\le 2e^{-2t^2}$ for every $n$, which matches the leading term of the limiting Kolmogorov law.\nIt is a common tool in learning theory and in the analysis of nonparametric estimators, because it is non-asymptotic.\n",
  "commit": "ff8a40749dc16545e3d2fc637f6b0196e81d0bfa",
  "front": {
    "about": [
      "A05WR6TEK5"
    ],
    "kind": "statement",
    "msc": [
      "62G30",
      "62G15",
      "60F10"
    ],
    "name": "Dvoretzky–Kiefer–Wolfowitz inequality",
    "role": "theorem",
    "same_as": {
      "wikidata": "Q5317822",
      "wikipedia": "Dvoretzky–Kiefer–Wolfowitz inequality"
    },
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/Dvoretzky%E2%80%93Kiefer%E2%80%93Wolfowitz_inequality"
      }
    ],
    "tag": "H5G7CNYZEB",
    "uses": [
      "A05WR6TEK5",
      "0AJN7WHBJZ",
      "0MXMS3AE13"
    ]
  },
  "kind": "statement",
  "link": "https://tamnd.github.io/math2/node/H5G7CNYZEB/",
  "links": [
    {
      "field": "about",
      "target": "A05WR6TEK5"
    },
    {
      "field": "uses",
      "target": "A05WR6TEK5"
    },
    {
      "field": "uses",
      "target": "0AJN7WHBJZ"
    },
    {
      "field": "uses",
      "target": "0MXMS3AE13"
    }
  ],
  "markdown": "https://tamnd.github.io/math2/node/H5G7CNYZEB.md",
  "msc": [
    "62G30",
    "62G15",
    "60F10"
  ],
  "name": "Dvoretzky–Kiefer–Wolfowitz inequality",
  "prov": null,
  "role": "theorem",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/H5G/H5G7CNYZEB.md",
  "status": null,
  "tag": "H5G7CNYZEB",
  "uri": "m2://H5G7CNYZEB",
  "url": "https://tamnd.github.io/math2/wiki/dvoretzky-kiefer-wolfowitz-inequality/",
  "writing": {
    "tag": "H5G7CNYZEB",
    "kind": "statement",
    "level": "W0",
    "next": [
      {
        "what": "an inventory row",
        "note": "the row of the inventory that the node accounts for",
        "have": 0,
        "want": 1
      },
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      }
    ]
  }
}