{
  "aliases": null,
  "backlinks": null,
  "body": "\n## Statement\n\nLet $X, Y$ be [[Hilbert spaces|HM2HHE3J51]], let $A \\colon X \\to Y$ be a bounded linear operator with adjoint $A^*$, and let $A^\\dagger$ be its [[Moore–Penrose generalized inverse|0YY8VX28JR]].\nLet $0 \u003c \\omega \u003c 2/\\norm{A}^2$.\nFor $z \\in Y$, the *Landweber iteration* is $x_0(z) = 0$ and\n$$x_{k+1}(z) = x_k(z) + \\omega A^* \\bigl( z - A x_k(z) \\bigr), \\qquad k \\ge 0.$$\n\n1. If $y \\in \\mathcal{R}(A) + \\mathcal{R}(A)^\\perp$, then $x_k(y) \\to A^\\dagger y$ as $k \\to \\infty$.\n2. If moreover $\\omega \\le 1/\\norm{A}^2$, then $\\norm{x_k(y^\\delta) - x_k(y)} \\le \\sqrt{\\omega k} \\, \\delta$ for all $y^\\delta \\in Y$ with $\\norm{y^\\delta - y} \\le \\delta$.\n3. Consequently, if $k(\\delta) \\to \\infty$ and $\\delta \\sqrt{k(\\delta)} \\to 0$ as $\\delta \\to 0$, then $x_{k(\\delta)}(y^\\delta) \\to A^\\dagger y$ uniformly over all $y^\\delta$ with $\\norm{y^\\delta - y} \\le \\delta$.\n\n## Remarks\n\nWrite $x^\\dagger = A^\\dagger y$; then $A^* y = A^* A x^\\dagger$.\nThe error satisfies $x_k(y) - x^\\dagger = -(I - \\omega A^* A)^k x^\\dagger$.\nThe self-adjoint operator $A^* A$ has spectrum in $[0, \\norm{A}^2]$, and $\\abs{1 - \\omega t} \u003c 1$ for $0 \u003c t \\le \\norm{A}^2$.\nSince $x^\\dagger$ is orthogonal to $\\ker A$, the spectral theorem gives $(I - \\omega A^* A)^k x^\\dagger \\to 0$, which is part 1.\nFor part 2, $x_k(z) = \\sum_{j \u003c k} (I - \\omega A^* A)^j \\omega A^* z$, and a spectral bound on $\\sqrt{\\omega t} \\sum_{j\u003ck} (1 - \\omega t)^j$ gives the factor $\\sqrt{\\omega k}$.\n\nThe iteration is gradient descent with fixed step $\\omega$ for $\\tfrac12 \\norm{A x - z}^2$.\nWhen $\\mathcal{R}(A)$ is not closed, $x_k(y^\\delta)$ first approaches $x^\\dagger$ and then diverges, a behavior called semi-convergence.\nSo the iteration index plays the role of the [[regularization|WTY54BFC46]] parameter, and one must stop early, for example by the discrepancy principle.\nWithout the condition $\\omega \u003c 2/\\norm{A}^2$, components along large singular values are amplified and the iteration diverges.\n\n## Explanations\n\n### L1 human\n\nLandweber iteration solves $A x = y$ by repeatedly correcting the guess in the direction that reduces the misfit $\\norm{A x - y}$.\nWith exact data it converges to the best solution.\nWith noisy data it first improves and later starts to fit the noise, so one stops it at the right moment.\n\n### L2 human\n\nLandweber iteration is the simplest iterative regularization method, and it is the model for others such as conjugate gradients on the normal equations.\nStopping after $k$ steps acts like a filter on the singular values: components with $\\omega \\sigma^2 k \\gg 1$ are recovered and the others are damped.\nIt converges slowly, but it extends directly to nonlinear operators and to constraints by adding a projection.\nIt was introduced by Louis Landweber in 1951.\n",
  "commit": "ff8a40749dc16545e3d2fc637f6b0196e81d0bfa",
  "front": {
    "about": [
      "WTY54BFC46"
    ],
    "kind": "statement",
    "msc": [
      "65J20",
      "65F22",
      "47A52"
    ],
    "name": "Landweber iteration",
    "role": "theorem",
    "same_as": {
      "wikidata": "Q6485285",
      "wikipedia": "Landweber iteration"
    },
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/Landweber_iteration"
      }
    ],
    "tag": "87DV5ZTJ9J",
    "uses": [
      "HM2HHE3J51",
      "0YY8VX28JR",
      "06EKW1AKXF",
      "WTY54BFC46"
    ]
  },
  "kind": "statement",
  "link": "https://tamnd.github.io/math2/node/87DV5ZTJ9J/",
  "links": [
    {
      "field": "about",
      "target": "WTY54BFC46"
    },
    {
      "field": "uses",
      "target": "HM2HHE3J51"
    },
    {
      "field": "uses",
      "target": "0YY8VX28JR"
    },
    {
      "field": "uses",
      "target": "06EKW1AKXF"
    },
    {
      "field": "uses",
      "target": "WTY54BFC46"
    }
  ],
  "markdown": "https://tamnd.github.io/math2/node/87DV5ZTJ9J.md",
  "msc": [
    "65J20",
    "65F22",
    "47A52"
  ],
  "name": "Landweber iteration",
  "prov": null,
  "role": "theorem",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/87D/87DV5ZTJ9J.md",
  "status": null,
  "tag": "87DV5ZTJ9J",
  "uri": "m2://87DV5ZTJ9J",
  "url": "https://tamnd.github.io/math2/wiki/landweber-iteration/",
  "writing": {
    "tag": "87DV5ZTJ9J",
    "kind": "statement",
    "level": "W0",
    "next": [
      {
        "what": "an inventory row",
        "note": "the row of the inventory that the node accounts for",
        "have": 0,
        "want": 1
      },
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      }
    ]
  }
}