{
  "aliases": null,
  "backlinks": null,
  "body": "\n## Definition\n\nLet $(X, \\mathcal{B}, \\mu)$ be a [[probability space|0NFPJVJJHX]] and let $T \\colon X \\to X$ be a [[measure-preserving transformation|556WBDAHNP]].\nA finite measurable partition is a finite [[partition|08QMKTWF9T]] $\\mathcal{P}$ of $X$ into sets of $\\mathcal{B}$.\nIts entropy is\n$$H(\\mathcal{P}) = -\\sum_{P \\in \\mathcal{P}} \\mu(P) \\log \\mu(P),$$\nwith the convention $0 \\log 0 = 0$.\nFor partitions $\\mathcal{P}$ and $\\mathcal{Q}$, the join $\\mathcal{P} \\vee \\mathcal{Q}$ is the partition into the nonempty sets $P \\cap Q$ with $P \\in \\mathcal{P}$ and $Q \\in \\mathcal{Q}$, their common [[refinement|0SPZ1H3N9V]].\nLet $T^{-i}\\mathcal{P}$ be the partition into the sets $T^{-i}P$ with $P \\in \\mathcal{P}$.\n\n1. The entropy of $T$ with respect to $\\mathcal{P}$ is\n$$h(T, \\mathcal{P}) = \\lim_{n \\to \\infty} \\frac{1}{n} H\\Big(\\bigvee_{i=0}^{n-1} T^{-i} \\mathcal{P}\\Big).$$\n2. The *Kolmogorov–Sinai entropy* of $T$ is $h_\\mu(T) = \\sup_{\\mathcal{P}} h(T, \\mathcal{P})$, over all finite measurable partitions $\\mathcal{P}$; it lies in $[0, \\infty]$.\n\n## Remarks\n\nThe limit in clause 1 exists because the sequence $a_n = H(\\bigvee_{i=0}^{n-1} T^{-i} \\mathcal{P})$ is subadditive, $a_{m+n} \\le a_m + a_n$.\n\nExample: let $T(x) = 2x \\bmod 1$ on $[0,1)$ with Lebesgue measure, and $\\mathcal{P} = \\set{[0, 1/2), [1/2, 1)}$.\nThe join $\\bigvee_{i=0}^{n-1} T^{-i} \\mathcal{P}$ is the partition into the $2^n$ dyadic intervals of length $2^{-n}$, so its entropy is $n \\log 2$, and $h(T, \\mathcal{P}) = \\log 2$.\nThis partition generates the Borel sets under $T$, and the Kolmogorov–Sinai generator theorem gives $h_\\mu(T) = \\log 2$.\n\nNon-example of positive entropy: the identity map has $\\bigvee_{i=0}^{n-1} T^{-i} \\mathcal{P} = \\mathcal{P}$, so $h(T, \\mathcal{P}) = \\lim H(\\mathcal{P})/n = 0$ for every $\\mathcal{P}$, and $h_\\mu(\\mathrm{id}) = 0$.\nRotations of the circle also have entropy $0$.\n\nIsomorphic systems have the same entropy.\nWith the natural logarithm the unit is the nat; with $\\log_2$ it is the bit.\n",
  "commit": "ff8a40749dc16545e3d2fc637f6b0196e81d0bfa",
  "front": {
    "defines": "X1D1C7VH6P",
    "kind": "definition",
    "msc": [
      "37A35",
      "28D20"
    ],
    "name": "Kolmogorov–Sinai entropy",
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/Measure-preserving_dynamical_system"
      }
    ],
    "tag": "Y41SH8BJVT",
    "uses": [
      "0NFPJVJJHX",
      "556WBDAHNP",
      "08QMKTWF9T",
      "0SPZ1H3N9V"
    ]
  },
  "kind": "definition",
  "link": "https://tamnd.github.io/math2/node/Y41SH8BJVT/",
  "links": [
    {
      "field": "defines",
      "target": "X1D1C7VH6P"
    },
    {
      "field": "uses",
      "target": "0NFPJVJJHX"
    },
    {
      "field": "uses",
      "target": "556WBDAHNP"
    },
    {
      "field": "uses",
      "target": "08QMKTWF9T"
    },
    {
      "field": "uses",
      "target": "0SPZ1H3N9V"
    }
  ],
  "markdown": "https://tamnd.github.io/math2/node/Y41SH8BJVT.md",
  "msc": [
    "37A35",
    "28D20"
  ],
  "name": "Kolmogorov–Sinai entropy",
  "prov": null,
  "role": "",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/Y41/Y41SH8BJVT.md",
  "status": null,
  "tag": "Y41SH8BJVT",
  "uri": "m2://Y41SH8BJVT",
  "url": "https://tamnd.github.io/math2/wiki/kolmogorov-sinai-entropy-definition/",
  "writing": {
    "tag": "Y41SH8BJVT",
    "kind": "definition",
    "level": "W0",
    "next": [
      {
        "what": "an inventory row",
        "note": "the row of the inventory that the node accounts for",
        "have": 0,
        "want": 1
      },
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      }
    ]
  }
}