{
  "aliases": null,
  "backlinks": null,
  "body": "\n## Statement\n\nLet $x_1, \\ldots, x_n$ be numbers in a binary [[floating-point|RTKRSC3KPH]] system with rounding to nearest and unit roundoff $u$ (see [[machine epsilon|JA3QXVF2B8]]).\nAssume that no overflow or underflow occurs.\nThe *Kahan summation algorithm* computes $\\hat s$ as follows, in floating-point operations:\nstart with $s = 0$ and $c = 0$; for $i = 1, \\ldots, n$ set $y = x_i - c$, $t = s + y$, $c = (t - s) - y$, $s = t$; finally $\\hat s = s$.\nThen\n$$\\hat s = \\sum_{i=1}^{n} (1 + \\mu_i) x_i \\quad\\text{with}\\quad \\abs{\\mu_i} \\le 2u + O(n u^2),$$\nand hence\n$$\\Bigl| \\hat s - \\sum_{i=1}^{n} x_i \\Bigr| \\le \\bigl( 2u + O(n u^2) \\bigr) \\sum_{i=1}^{n} \\abs{x_i}.$$\n\n## Remarks\n\nThe variable $c$ holds the low-order part lost when $y$ is added to the large running sum $s$.\nThe difference $t - s$ recovers what was actually added, and subtracting $y$ gives minus the [[round-off error|CPN6SD3BTV]] of the addition.\nThis correction is fed back into the next term.\nThe proof tracks the rounding errors of the four operations through the loop; the key point is that, in binary arithmetic, $c$ captures the rounding error of $s + y$ almost exactly.\nFor comparison, plain recursive summation only gives the bound $\\frac{(n-1)u}{1 - (n-1)u} \\sum \\abs{x_i}$, which grows linearly in $n$.\nThe result is a [[backward error|CMXHRNWY35]] bound: $\\hat s$ is the exact sum of the slightly perturbed terms $(1 + \\mu_i) x_i$.\nThe relative error $\\abs{\\hat s - s}/\\abs{s}$ can still be large when $\\sum \\abs{x_i} \\gg \\abs{\\sum x_i}$, that is, when the sum is ill-conditioned.\nOptimizing compilers that treat floating-point addition as associative can simplify $c$ to $0$ and destroy the method.\n\n## Explanations\n\n### L1 human\n\nAdding many numbers on a computer loses a little accuracy at each addition, and these losses can add up.\nKahan's trick keeps a second variable that remembers the small part lost at each step and adds it back in the next step.\nWith this, the error no longer grows with the number of terms: it stays about twice the error of a single rounding, as long as $n u$ is small.\n\n### L2 human\n\nThe bound should be compared with the bound $\\gamma_{n-1} \\sum \\abs{x_i}$, $\\gamma_k = ku/(1 - ku)$, for recursive summation and $\\gamma_{\\lceil \\log_2 n \\rceil} \\sum \\abs{x_i}$ for [[pairwise summation|SVSVMWA9NQ]].\nCompensated summation costs four floating-point operations per term instead of one, but removes the dependence on $n$ from the leading term.\nIt is the simplest example of an error-free transformation: the pair $(t, -c)$ represents $s + y$ exactly or nearly exactly.\nThe same idea underlies compensated algorithms for dot products and polynomial evaluation.\n",
  "commit": "ff8a40749dc16545e3d2fc637f6b0196e81d0bfa",
  "front": {
    "about": [
      "CPN6SD3BTV"
    ],
    "kind": "statement",
    "msc": [
      "65G50",
      "65B10"
    ],
    "name": "Kahan summation algorithm",
    "role": "theorem",
    "same_as": {
      "wikidata": "Q2982293",
      "wikipedia": "Kahan summation algorithm"
    },
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/Kahan_summation_algorithm"
      }
    ],
    "tag": "4JAEKN22Y5",
    "uses": [
      "RTKRSC3KPH",
      "JA3QXVF2B8",
      "CPN6SD3BTV",
      "CMXHRNWY35"
    ]
  },
  "kind": "statement",
  "link": "https://tamnd.github.io/math2/node/4JAEKN22Y5/",
  "links": [
    {
      "field": "about",
      "target": "CPN6SD3BTV"
    },
    {
      "field": "uses",
      "target": "RTKRSC3KPH"
    },
    {
      "field": "uses",
      "target": "JA3QXVF2B8"
    },
    {
      "field": "uses",
      "target": "CPN6SD3BTV"
    },
    {
      "field": "uses",
      "target": "CMXHRNWY35"
    }
  ],
  "markdown": "https://tamnd.github.io/math2/node/4JAEKN22Y5.md",
  "msc": [
    "65G50",
    "65B10"
  ],
  "name": "Kahan summation algorithm",
  "prov": null,
  "role": "theorem",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/4JA/4JAEKN22Y5.md",
  "status": null,
  "tag": "4JAEKN22Y5",
  "uri": "m2://4JAEKN22Y5",
  "url": "https://tamnd.github.io/math2/wiki/kahan-summation-algorithm/",
  "writing": {
    "tag": "4JAEKN22Y5",
    "kind": "statement",
    "level": "W0",
    "next": [
      {
        "what": "an inventory row",
        "note": "the row of the inventory that the node accounts for",
        "have": 0,
        "want": 1
      },
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      }
    ]
  }
}