{
  "aliases": null,
  "backlinks": null,
  "body": "\n## Definition\n\nLet $(X, \\rho)$ be a [[metric space|0VKD1TP7F3]], let $K \\subseteq X$ be a nonempty [[totally bounded set|020Q06TYYH]], and let $\\varepsilon \u003e 0$.\nLet $N_\\varepsilon(K)$ be the least number $N$ such that $K \\subseteq A_1 \\cup \\cdots \\cup A_N$ for some subsets $A_1, \\dots, A_N$ of $X$ of diameter at most $2\\varepsilon$.\nThe *Kolmogorov $\\varepsilon$-entropy* of $K$ is\n$$H_\\varepsilon(K) = \\log_2 N_\\varepsilon(K).$$\n\n## Remarks\n\nThe number $N_\\varepsilon(K)$ is finite because $K$ is totally bounded.\nCovering by closed balls of radius $\\varepsilon$ instead gives the covering number $N'_\\varepsilon(K)$, and $N_\\varepsilon(K) \\le N'_\\varepsilon(K) \\le N_{\\varepsilon/2}(K)$.\nSo the two versions have the same growth up to a change of $\\varepsilon$ by a factor $2$.\nKolmogorov and Tikhomirov also used the $\\varepsilon$-capacity, the logarithm of the largest number of points of $K$ with pairwise distances greater than $\\varepsilon$.\n\nExample: $K = [0, 1]$ in $\\R$.\nA set of diameter at most $2\\varepsilon$ lies in an interval of length $2\\varepsilon$, so at least $1/(2\\varepsilon)$ of them are needed.\nOn the other hand, $\\ceil{1/(2\\varepsilon)}$ consecutive closed intervals of length $2\\varepsilon$ suffice.\nSo $N_\\varepsilon([0, 1]) = \\ceil{1/(2\\varepsilon)}$ and $H_\\varepsilon([0, 1]) = \\log_2 \\ceil{1/(2\\varepsilon)}$.\n\nNon-example: $K = [0, \\infty)$ is not totally bounded, and no finite family of sets of diameter $2\\varepsilon$ covers it, so $H_\\varepsilon(K)$ is not defined, or is $+\\infty$.\n\nFor the class of functions $f$ on $[0, 1]$ with $\\abs{f} \\le 1$ and Lipschitz constant at most $1$, in the supremum norm, $H_\\varepsilon$ is of order $1/\\varepsilon$ as $\\varepsilon \\to 0$.\n",
  "commit": "ff8a40749dc16545e3d2fc637f6b0196e81d0bfa",
  "front": {
    "defines": "63XHDVA5FS",
    "kind": "definition",
    "msc": [
      "41A46",
      "46B50"
    ],
    "name": "Kolmogorov epsilon-entropy",
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/Covering_number"
      }
    ],
    "tag": "9ZK07AKD4M",
    "uses": [
      "0VKD1TP7F3",
      "020Q06TYYH"
    ]
  },
  "kind": "definition",
  "link": "https://tamnd.github.io/math2/node/9ZK07AKD4M/",
  "links": [
    {
      "field": "defines",
      "target": "63XHDVA5FS"
    },
    {
      "field": "uses",
      "target": "0VKD1TP7F3"
    },
    {
      "field": "uses",
      "target": "020Q06TYYH"
    }
  ],
  "markdown": "https://tamnd.github.io/math2/node/9ZK07AKD4M.md",
  "msc": [
    "41A46",
    "46B50"
  ],
  "name": "Kolmogorov epsilon-entropy",
  "prov": null,
  "role": "",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/9ZK/9ZK07AKD4M.md",
  "status": null,
  "tag": "9ZK07AKD4M",
  "uri": "m2://9ZK07AKD4M",
  "url": "https://tamnd.github.io/math2/wiki/kolmogorov-epsilon-entropy-definition/",
  "writing": {
    "tag": "9ZK07AKD4M",
    "kind": "definition",
    "level": "W0",
    "next": [
      {
        "what": "an inventory row",
        "note": "the row of the inventory that the node accounts for",
        "have": 0,
        "want": 1
      },
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      }
    ]
  }
}