{
  "aliases": null,
  "backlinks": null,
  "body": "\n## Statement\n\nLet $A$ be an [[invertible|0EB9N0S33C]] $n \\times n$ real or complex matrix, and let $b$ be a vector.\nLet $x = A^{-1} b$ be the solution of the [[linear system|0GT4WWGWH2]] $Ax = b$.\nFix a vector norm and its [[operator norm|06EKW1AKXF]], and a number $\\eta$ with $0 \\le \\eta \u003c 1/2$.\n*Iterative refinement* starts from an approximation $x_0$ and repeats, for $m = 0, 1, 2, \\dots$:\n\n1. compute the residual $r_m = b - A x_m$;\n2. compute a correction $d_m$ with $(A + E_m) d_m = r_m$, for some matrix $E_m$ with $\\norm{A^{-1}} \\norm{E_m} \\le \\eta$;\n3. set $x_{m+1} = x_m + d_m$.\n\nThen, if the residuals and updates are computed exactly,\n$$\\norm{x_{m+1} - x} \\le \\frac{\\eta}{1 - \\eta} \\norm{x_m - x}$$\nfor all $m$, so $x_m \\to x$ linearly with rate at most $\\eta / (1 - \\eta) \u003c 1$.\n\n## Remarks\n\nLet $e_m = x - x_m$, so that $r_m = A e_m$.\nThen $e_{m+1} = e_m - (A + E_m)^{-1} A e_m = (A + E_m)^{-1} E_m e_m$.\nSince $\\norm{A^{-1} E_m} \\le \\eta \u003c 1$, the matrix $A + E_m = A(I + A^{-1} E_m)$ is invertible with $\\norm{(A + E_m)^{-1}} \\le \\norm{A^{-1}} / (1 - \\eta)$, which gives the bound.\nThe matrix $E_m$ models the error of an inexact solver, such as a computed [[LU decomposition|0RKG5YW66Y]] reused at each step, or a solve in lower precision.\nIf the solver is too inaccurate, so that $\\eta \\ge 1/2$, the bound gives no contraction and the process can diverge.\nThis happens when the [[condition number|AB3S6MHJ8X]] of $A$ times the solver's relative accuracy is not small.\nIn floating-point practice, the residual is the critical step: computing $r_m$ in higher precision lets refinement reach a solution accurate to the working precision, even for fairly ill-conditioned $A$.\nIn exact arithmetic with $E_m = 0$, one step gives the exact solution.\n\n## Explanations\n\n### L1 human\n\nSolve the system approximately, then check how far off you are by computing the residual $b - Ax$.\nThe residual is itself the right-hand side of a system for the error, so solve that system approximately too, and add the correction.\nEach round removes most of the remaining error, as long as the solver is not too crude.\n\n### L2 human\n\nIterative refinement turns a cheap or low-precision solver into an accurate one.\nIt reuses one factorization of $A$, so each extra round costs only $O(n^2)$ operations after the $O(n^3)$ factorization.\nModern mixed-precision solvers factor in low precision and refine in higher precision to get both speed and accuracy.\n",
  "commit": "ff8a40749dc16545e3d2fc637f6b0196e81d0bfa",
  "front": {
    "about": [
      "0GT4WWGWH2"
    ],
    "kind": "statement",
    "msc": [
      "65F05",
      "65G50"
    ],
    "name": "Iterative refinement",
    "role": "proposition",
    "same_as": {
      "wikidata": "Q6094420",
      "wikipedia": "Iterative refinement"
    },
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/Iterative_refinement"
      }
    ],
    "tag": "QTVP4WP1XX",
    "uses": [
      "0GT4WWGWH2",
      "0EB9N0S33C",
      "06EKW1AKXF",
      "AB3S6MHJ8X",
      "0RKG5YW66Y"
    ]
  },
  "kind": "statement",
  "link": "https://tamnd.github.io/math2/node/QTVP4WP1XX/",
  "links": [
    {
      "field": "about",
      "target": "0GT4WWGWH2"
    },
    {
      "field": "uses",
      "target": "0GT4WWGWH2"
    },
    {
      "field": "uses",
      "target": "0EB9N0S33C"
    },
    {
      "field": "uses",
      "target": "06EKW1AKXF"
    },
    {
      "field": "uses",
      "target": "AB3S6MHJ8X"
    },
    {
      "field": "uses",
      "target": "0RKG5YW66Y"
    }
  ],
  "markdown": "https://tamnd.github.io/math2/node/QTVP4WP1XX.md",
  "msc": [
    "65F05",
    "65G50"
  ],
  "name": "Iterative refinement",
  "prov": null,
  "role": "proposition",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/QTV/QTVP4WP1XX.md",
  "status": null,
  "tag": "QTVP4WP1XX",
  "uri": "m2://QTVP4WP1XX",
  "url": "https://tamnd.github.io/math2/wiki/iterative-refinement/",
  "writing": {
    "tag": "QTVP4WP1XX",
    "kind": "statement",
    "level": "W0",
    "next": [
      {
        "what": "an inventory row",
        "note": "the row of the inventory that the node accounts for",
        "have": 0,
        "want": 1
      },
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      }
    ]
  }
}