{
  "aliases": null,
  "backlinks": [
    {
      "field": "uses",
      "target": "1SYY57DCWT"
    },
    {
      "field": "uses",
      "target": "A3FR6PNEPB"
    }
  ],
  "body": "\n## Statement\n\nLet $A$ be a complex $n \\times n$ matrix and let $b \\in \\C^n$ be nonzero.\nThe *Arnoldi iteration* sets $q_1 = b / \\norm{b}$ and, for $j = 1, 2, \\dots$, computes\n$$h_{ij} = q_i^* A q_j \\ (1 \\le i \\le j), \\quad w_j = A q_j - \\sum_{i=1}^{j} h_{ij} q_i, \\quad h_{j+1, j} = \\norm{w_j}, \\quad q_{j+1} = w_j / h_{j+1, j}.$$\nHere $\\norm{\\cdot}$ is the Euclidean norm.\nSuppose $h_{j+1, j} \\ne 0$ for $j = 1, \\dots, m$.\nLet $Q_m$ be the $n \\times m$ matrix with columns $q_1, \\dots, q_m$.\nLet $\\tilde{H}_m$ be the $(m+1) \\times m$ matrix of the $h_{ij}$, with $h_{ij} = 0$ for $i \u003e j + 1$, and let $H_m$ be its first $m$ rows.\nThen:\n\n1. $q_1, \\dots, q_{m+1}$ is an orthonormal basis of the [[Krylov subspace|QB45PCGQPV]] $\\mathcal{K}_{m+1}(A, b)$;\n2. $A Q_m = Q_{m+1} \\tilde{H}_m$, and $H_m = Q_m^* A Q_m$ is an upper [[Hessenberg matrix|YCEEWJTX56]];\n3. if $h_{m+1, m} = 0$ instead, the iteration stops at step $m$, $\\mathcal{K}_m(A, b)$ is invariant under $A$, and every eigenvalue of $H_m$ is an eigenvalue of $A$.\n\n## Remarks\n\nThe iteration is the [[Gram–Schmidt process|0VT6P0GNQR]], in its modified form in practice, applied to the vectors $b, A q_1, A q_2, \\dots$.\nSince $A q_j$ lies in $\\mathcal{K}_{j+1}(A, b)$, clause 1 follows by induction, and clause 2 is the matrix form of the defining recurrence.\nClause 3 holds because $A Q_m = Q_m H_m$ when $h_{m+1, m} = 0$.\nThe [[eigenvalues|0BQAA2BK3Q]] of $H_m$ are the Ritz values; the extreme ones often approximate eigenvalues of $A$ well for $m$ much smaller than $n$.\nExample: if $b$ is an eigenvector of $A$ for $\\lambda$, then $w_1 = 0$, the iteration stops at $m = 1$, and $H_1 = (\\lambda)$.\nIf $A$ is [[Hermitian|0P17NGQQMQ]], then $H_m$ is Hermitian and Hessenberg, hence [[tridiagonal|DARX1KZRHZ]].\nThen the recurrence has only three terms, and it becomes the [[Lanczos algorithm|7RRB4PCRJB]].\n\n## Explanations\n\n### L1 human\n\nThe Arnoldi iteration builds an orthonormal basis of the space spanned by $b, A b, A^2 b, \\dots$, one vector at a time.\nIn this basis, the action of $A$ looks like a small matrix that is almost triangular.\nStudying this small matrix reveals information about the large one, such as its extreme eigenvalues.\n\n### L2 human\n\nArnoldi is the engine of GMRES for nonsymmetric linear systems and of the implicitly restarted Arnoldi method for eigenvalues.\nIts cost grows with $m$: step $j$ needs one product with $A$ and $j$ inner products, and all $q_j$ must be stored.\nThis is why practical codes restart after a fixed number of steps.\nThe relation $A Q_m = Q_{m+1} \\tilde{H}_m$ reduces large problems on $\\mathcal{K}_m$ to small problems with $\\tilde{H}_m$.\n",
  "commit": "ff8a40749dc16545e3d2fc637f6b0196e81d0bfa",
  "front": {
    "about": [
      "QB45PCGQPV",
      "YCEEWJTX56"
    ],
    "kind": "statement",
    "msc": [
      "65F15",
      "65F10",
      "65F25"
    ],
    "name": "Arnoldi iteration",
    "role": "proposition",
    "same_as": {
      "wikidata": "Q696822",
      "wikipedia": "Arnoldi iteration"
    },
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/Arnoldi_iteration"
      }
    ],
    "tag": "CJ5EC820S7",
    "uses": [
      "QB45PCGQPV",
      "YCEEWJTX56",
      "0VT6P0GNQR",
      "0P17NGQQMQ",
      "DARX1KZRHZ",
      "0BQAA2BK3Q",
      "7RRB4PCRJB"
    ]
  },
  "kind": "statement",
  "link": "https://tamnd.github.io/math2/node/CJ5EC820S7/",
  "links": [
    {
      "field": "about",
      "target": "QB45PCGQPV"
    },
    {
      "field": "about",
      "target": "YCEEWJTX56"
    },
    {
      "field": "uses",
      "target": "QB45PCGQPV"
    },
    {
      "field": "uses",
      "target": "YCEEWJTX56"
    },
    {
      "field": "uses",
      "target": "0VT6P0GNQR"
    },
    {
      "field": "uses",
      "target": "0P17NGQQMQ"
    },
    {
      "field": "uses",
      "target": "DARX1KZRHZ"
    },
    {
      "field": "uses",
      "target": "0BQAA2BK3Q"
    },
    {
      "field": "uses",
      "target": "7RRB4PCRJB"
    }
  ],
  "markdown": "https://tamnd.github.io/math2/node/CJ5EC820S7.md",
  "msc": [
    "65F15",
    "65F10",
    "65F25"
  ],
  "name": "Arnoldi iteration",
  "prov": null,
  "role": "proposition",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/CJ5/CJ5EC820S7.md",
  "status": null,
  "tag": "CJ5EC820S7",
  "uri": "m2://CJ5EC820S7",
  "url": "https://tamnd.github.io/math2/wiki/arnoldi-iteration/",
  "writing": {
    "tag": "CJ5EC820S7",
    "kind": "statement",
    "level": "W0",
    "next": [
      {
        "what": "an inventory row",
        "note": "the row of the inventory that the node accounts for",
        "have": 0,
        "want": 1
      },
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      }
    ]
  }
}