{
  "aliases": null,
  "backlinks": [
    {
      "field": "uses",
      "target": "24YCKE2RPH"
    }
  ],
  "body": "\n## Statement\n\nLet $A$ be a real [[symmetric|0VR671PHE5]] [[invertible|0EB9N0S33C]] $n \\times n$ matrix, possibly indefinite, let $b \\in \\R^n$, and let $x_0 \\in \\R^n$ with $r_0 = b - A x_0$.\nFor $k \\ge 1$, the $k$-th iterate of the *minimal residual method* (MINRES) is the vector $x_k \\in x_0 + \\mathcal{K}_k(A, r_0)$ that minimizes $\\norm{b - A x}_2$ there.\nHere $\\mathcal{K}_k$ is the [[Krylov subspace|QB45PCGQPV]].\n\n1. The iterates can be computed by recurrences of fixed length. Each step costs one product with $A$ and a number of vector operations that does not depend on $k$.\n2. Let $r_k = b - A x_k$, and let $P_k$ be the set of polynomials $p$ of degree at most $k$ with $p(0) = 1$. Then $$\\norm{r_k}_2 = \\min_{p \\in P_k} \\norm{p(A) r_0}_2 \\le \\min_{p \\in P_k} \\max_{\\lambda} \\abs{p(\\lambda)} \\, \\norm{r_0}_2,$$ where $\\lambda$ runs over the eigenvalues of $A$.\n3. In exact arithmetic some $r_k$ with $k \\le n$ is zero.\n\n## Remarks\n\nThe [[Lanczos algorithm|7RRB4PCRJB]] gives an orthonormal basis $Q_{k+1}$ of $\\mathcal{K}_{k+1}(A, r_0)$ with $A Q_k = Q_{k+1} \\tilde{T}_k$, where $\\tilde{T}_k$ is $(k+1) \\times k$ and [[tridiagonal|DARX1KZRHZ]].\nThe minimization reduces to a small least squares problem with $\\tilde{T}_k$.\nA QR factorization of $\\tilde{T}_k$ by Givens rotations can be updated with each new column, and this yields the short recurrences of clause 1.\nClause 2 uses that $A$ is orthogonally diagonalizable, so $\\norm{p(A)}_2 = \\max_\\lambda \\abs{p(\\lambda)}$.\nExample: if $A$ has only the [[eigenvalues|0BQAA2BK3Q]] $1$ and $-1$, then $p(t) = 1 - t^2$ gives $r_2 = 0$.\nFor a nonsymmetric $A$ the Lanczos recurrence no longer produces an orthonormal basis, and one needs [[GMRES|A3FR6PNEPB]] with its long recurrences.\n\n## Explanations\n\n### L1 human\n\nMINRES solves symmetric systems that need not be positive definite, such as saddle point problems.\nAt each step it picks, among the vectors it can reach with the matrix, the one that makes the residual as small as possible.\nThanks to symmetry, it only needs to remember a few vectors.\n\n### L2 human\n\nMINRES is mathematically [[GMRES|A3FR6PNEPB]] specialized to symmetric matrices, where the Arnoldi process becomes the Lanczos process.\nIt fills the gap left by the [[conjugate gradient method|AFXZGQB4EN]], which can break down when $A$ is indefinite.\nFor symmetric positive definite $A$ both apply: conjugate gradients minimize the energy norm of the error, MINRES the Euclidean norm of the residual.\nLike other Krylov [[iterative methods|2RPE77ACER]], it is used with a preconditioner, which must be symmetric positive definite to keep the symmetry.\nThe method is due to Paige and Saunders.\n",
  "commit": "ff8a40749dc16545e3d2fc637f6b0196e81d0bfa",
  "front": {
    "about": [
      "2RPE77ACER",
      "QB45PCGQPV"
    ],
    "kind": "statement",
    "msc": [
      "65F10"
    ],
    "name": "Minimal residual method",
    "role": "theorem",
    "same_as": {
      "wikidata": "Q1453478",
      "wikipedia": "Minimal residual method"
    },
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/Minimal_residual_method"
      }
    ],
    "tag": "3HRHSHTG9C",
    "uses": [
      "0VR671PHE5",
      "0EB9N0S33C",
      "QB45PCGQPV",
      "7RRB4PCRJB",
      "DARX1KZRHZ",
      "0BQAA2BK3Q",
      "A3FR6PNEPB",
      "AFXZGQB4EN",
      "2RPE77ACER"
    ]
  },
  "kind": "statement",
  "link": "https://tamnd.github.io/math2/node/3HRHSHTG9C/",
  "links": [
    {
      "field": "about",
      "target": "2RPE77ACER"
    },
    {
      "field": "about",
      "target": "QB45PCGQPV"
    },
    {
      "field": "uses",
      "target": "0VR671PHE5"
    },
    {
      "field": "uses",
      "target": "0EB9N0S33C"
    },
    {
      "field": "uses",
      "target": "QB45PCGQPV"
    },
    {
      "field": "uses",
      "target": "7RRB4PCRJB"
    },
    {
      "field": "uses",
      "target": "DARX1KZRHZ"
    },
    {
      "field": "uses",
      "target": "0BQAA2BK3Q"
    },
    {
      "field": "uses",
      "target": "A3FR6PNEPB"
    },
    {
      "field": "uses",
      "target": "AFXZGQB4EN"
    },
    {
      "field": "uses",
      "target": "2RPE77ACER"
    }
  ],
  "markdown": "https://tamnd.github.io/math2/node/3HRHSHTG9C.md",
  "msc": [
    "65F10"
  ],
  "name": "Minimal residual method",
  "prov": null,
  "role": "theorem",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/3HR/3HRHSHTG9C.md",
  "status": null,
  "tag": "3HRHSHTG9C",
  "uri": "m2://3HRHSHTG9C",
  "url": "https://tamnd.github.io/math2/wiki/minimal-residual-method/",
  "writing": {
    "tag": "3HRHSHTG9C",
    "kind": "statement",
    "level": "W0",
    "next": [
      {
        "what": "an inventory row",
        "note": "the row of the inventory that the node accounts for",
        "have": 0,
        "want": 1
      },
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      }
    ]
  }
}