{
  "aliases": null,
  "backlinks": [
    {
      "field": "uses",
      "target": "3HRHSHTG9C"
    },
    {
      "field": "uses",
      "target": "MA64H1RTP1"
    }
  ],
  "body": "\n## Statement\n\nLet $A$ be an [[invertible|0EB9N0S33C]] complex $n \\times n$ matrix, let $b \\in \\C^n$, let $x_0 \\in \\C^n$, and let $r_0 = b - A x_0 \\ne 0$.\nFor $k \\ge 1$, the $k$-th iterate of the *generalized minimal residual method* (GMRES) is the vector $x_k \\in x_0 + \\mathcal{K}_k(A, r_0)$ that minimizes $\\norm{b - A x}_2$ there.\nHere $\\mathcal{K}_k$ is the [[Krylov subspace|QB45PCGQPV]].\n\n1. $x_k$ exists and is unique, and $\\norm{r_k}_2 \\le \\norm{r_{k-1}}_2$, where $r_k = b - A x_k$.\n2. Run $k$ steps of the [[Arnoldi iteration|CJ5EC820S7]] from $r_0$, giving $Q_{k+1}$ and $\\tilde{H}_k$, and let $\\beta = \\norm{r_0}_2$. Then $x_k = x_0 + Q_k y_k$, where $y_k$ minimizes $\\norm{\\beta e_1 - \\tilde{H}_k y}_2$ over $y \\in \\C^k$.\n3. Some $r_k$ with $k \\le n$ is zero, so GMRES finds $A^{-1} b$ in at most $n$ steps in exact arithmetic.\n4. Let $P_k$ be the set of polynomials $p$ of degree at most $k$ with $p(0) = 1$. If $A = V \\Lambda V^{-1}$ with $\\Lambda$ diagonal, then $$\\frac{\\norm{r_k}_2}{\\norm{r_0}_2} \\le \\kappa_2(V) \\min_{p \\in P_k} \\max_{\\lambda} \\abs{p(\\lambda)},$$ where $\\lambda$ runs over the eigenvalues of $A$.\n\n## Remarks\n\nEvery $x \\in x_0 + \\mathcal{K}_k(A, r_0)$ has residual $p(A) r_0$ for a polynomial $p$ of degree at most $k$ with $p(0) = 1$, which gives clause 4.\nClause 1 holds because the search spaces grow, and clause 2 follows from $A Q_k = Q_{k+1} \\tilde{H}_k$ and the orthonormality of the columns of $Q_{k+1}$.\nFor clause 3, if $\\mathcal{K}_k(A, r_0)$ becomes invariant under $A$, then it contains $A^{-1} r_0$ and the residual vanishes.\nClause 2 turns the large minimization into a small [[least squares|0BYTQ4X1HC]] problem with a Hessenberg matrix, solved by Givens rotations.\nExample: if $A = I + N$ with $N^2 = 0$, then $p(t) = (1 - t)^2$ vanishes on $A$, so $r_2 = 0$ for any $b$.\nThe constant $\\kappa_2(V)$ is the [[condition number|AB3S6MHJ8X]] of the eigenvector matrix; for normal $A$ it equals $1$.\nFor highly non-normal $A$, the eigenvalues alone do not decide the convergence.\n\n## Explanations\n\n### L1 human\n\nGMRES looks at all vectors it can build from the starting residual by repeated multiplication with $A$.\nAmong them it picks the one whose residual is as small as possible.\nThe residual can never grow, and with exact arithmetic the method reaches the solution after at most $n$ steps.\nIt works for any invertible matrix, symmetric or not.\n\n### L2 human\n\nGMRES is the residual-minimizing [[iterative method|2RPE77ACER]] for general nonsymmetric systems, while the [[conjugate gradient method|AFXZGQB4EN]] minimizes the energy norm for symmetric positive definite ones.\nSince Arnoldi needs all basis vectors, the cost per step grows; restarted GMRES($m$) throws them away every $m$ steps, and it may then stagnate.\nIt is almost always used with a [[preconditioner|KE98SW3SBD]], often an incomplete LU factorization.\nThe method is due to Saad and Schultz.\n",
  "commit": "ff8a40749dc16545e3d2fc637f6b0196e81d0bfa",
  "front": {
    "about": [
      "2RPE77ACER",
      "QB45PCGQPV"
    ],
    "kind": "statement",
    "msc": [
      "65F10"
    ],
    "name": "Generalized minimal residual method",
    "role": "theorem",
    "same_as": {
      "wikidata": "Q1432976",
      "wikipedia": "Generalized minimal residual method"
    },
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/Generalized_minimal_residual_method"
      }
    ],
    "tag": "A3FR6PNEPB",
    "uses": [
      "0EB9N0S33C",
      "QB45PCGQPV",
      "CJ5EC820S7",
      "0BYTQ4X1HC",
      "AB3S6MHJ8X",
      "0BQAA2BK3Q",
      "AFXZGQB4EN",
      "KE98SW3SBD",
      "2RPE77ACER"
    ]
  },
  "kind": "statement",
  "link": "https://tamnd.github.io/math2/node/A3FR6PNEPB/",
  "links": [
    {
      "field": "about",
      "target": "2RPE77ACER"
    },
    {
      "field": "about",
      "target": "QB45PCGQPV"
    },
    {
      "field": "uses",
      "target": "0EB9N0S33C"
    },
    {
      "field": "uses",
      "target": "QB45PCGQPV"
    },
    {
      "field": "uses",
      "target": "CJ5EC820S7"
    },
    {
      "field": "uses",
      "target": "0BYTQ4X1HC"
    },
    {
      "field": "uses",
      "target": "AB3S6MHJ8X"
    },
    {
      "field": "uses",
      "target": "0BQAA2BK3Q"
    },
    {
      "field": "uses",
      "target": "AFXZGQB4EN"
    },
    {
      "field": "uses",
      "target": "KE98SW3SBD"
    },
    {
      "field": "uses",
      "target": "2RPE77ACER"
    }
  ],
  "markdown": "https://tamnd.github.io/math2/node/A3FR6PNEPB.md",
  "msc": [
    "65F10"
  ],
  "name": "Generalized minimal residual method",
  "prov": null,
  "role": "theorem",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/A3F/A3FR6PNEPB.md",
  "status": null,
  "tag": "A3FR6PNEPB",
  "uri": "m2://A3FR6PNEPB",
  "url": "https://tamnd.github.io/math2/wiki/generalized-minimal-residual-method/",
  "writing": {
    "tag": "A3FR6PNEPB",
    "kind": "statement",
    "level": "W0",
    "next": [
      {
        "what": "an inventory row",
        "note": "the row of the inventory that the node accounts for",
        "have": 0,
        "want": 1
      },
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      }
    ]
  }
}