{
  "aliases": null,
  "backlinks": [
    {
      "field": "uses",
      "target": "6KQJR85ZC6"
    },
    {
      "field": "uses",
      "target": "AC9PY6FBYF"
    }
  ],
  "body": "\n## Statement\n\nLet $A$ be a real $n \\times n$ matrix with nonzero diagonal entries, and let $b \\in \\R^n$.\nWrite $A = L_* + U$, where $L_*$ is the lower triangular part of $A$ with the diagonal, and $U$ is the strictly upper triangular part.\nThe *Gauss–Seidel method* for the [[linear system|0GT4WWGWH2]] $A x = b$ starts from any $x^{(0)} \\in \\R^n$ and sets\n$$x^{(k+1)} = L_*^{-1} \\left( b - U x^{(k)} \\right), \\quad \\text{that is,} \\quad x^{(k+1)}_i = \\frac{1}{a_{ii}} \\Big( b_i - \\sum_{j \u003c i} a_{ij} x^{(k+1)}_j - \\sum_{j \u003e i} a_{ij} x^{(k)}_j \\Big).$$\nSuppose that one of the following holds:\n\n1. $A$ is strictly [[diagonally dominant|89P5GTKW87]] by rows;\n2. $A$ is symmetric and [[positive definite|0KX66CZMBB]].\n\nThen $A$ is invertible, and for every $b$ and every $x^{(0)}$ the iterates converge to $A^{-1} b$.\n\n## Remarks\n\nThe error satisfies $e_{k+1} = G e_k$ with $G = -L_*^{-1} U$, and convergence for all starts means that the [[spectral radius|9V7YQA85PC]] of $G$ is less than $1$.\nIn case 1, an eigenvector argument bounds each eigenvalue $\\lambda$ of $G$ by $\\abs{\\lambda} \\le \\max_i \\frac{\\sum_{j \u003e i} \\abs{a_{ij}}}{\\abs{a_{ii}} - \\sum_{j \u003c i} \\abs{a_{ij}}} \u003c 1$.\nIn case 2, the energy $\\norm{e}_A^2 = e^T A e$ strictly decreases along the iteration, by a computation with $L_*^T + L_* - A = D$, where $D$ is the positive diagonal of $A$.\nExample: for $A$ with rows $(2, 1)$ and $(1, 2)$ and $b = (3, 3)$, starting at $0$ gives $(1.5, 0.75)$, then $(1.125, 0.9375)$, tending to $(1, 1)$.\nThe error is multiplied by $1/4$ at each step, the square of the [[Jacobi|B8FVHNXAN1]] factor $1/2$.\nWithout the hypotheses the method can diverge: for $A$ with rows $(1, 2)$ and $(2, 1)$, which is symmetric but indefinite, $G$ has spectral radius $4$.\n\n## Explanations\n\n### L1 human\n\nGauss–Seidel is the [[Jacobi method|B8FVHNXAN1]] with one change: each new value is used as soon as it is computed.\nOne sweeps through the equations in order, and each equation sees the freshest values of the unknowns before it.\nThis usually converges faster than Jacobi and needs only one copy of the vector in memory.\n\n### L2 human\n\nIn matrix form Gauss–Seidel is the [[iterative method|2RPE77ACER]] of the splitting $A = M - N$ with $M = L_*$, so each step is a forward substitution.\nFor symmetric positive definite $A$ it is a coordinate-wise minimization of the energy $\\frac12 x^T A x - b^T x$.\nIts sweeps depend on the order of the unknowns, which makes it harder to parallelize than Jacobi.\nSuccessive over-relaxation adds a weight to each update and can converge much faster.\n",
  "commit": "ff8a40749dc16545e3d2fc637f6b0196e81d0bfa",
  "front": {
    "about": [
      "2RPE77ACER"
    ],
    "kind": "statement",
    "msc": [
      "65F10"
    ],
    "name": "Gauss–Seidel method",
    "role": "theorem",
    "same_as": {
      "wikidata": "Q1069090",
      "wikipedia": "Gauss–Seidel method"
    },
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/Gauss%E2%80%93Seidel_method"
      }
    ],
    "tag": "G0EDYEDG4S",
    "uses": [
      "0GT4WWGWH2",
      "89P5GTKW87",
      "0KX66CZMBB",
      "9V7YQA85PC",
      "2RPE77ACER",
      "B8FVHNXAN1"
    ]
  },
  "kind": "statement",
  "link": "https://tamnd.github.io/math2/node/G0EDYEDG4S/",
  "links": [
    {
      "field": "about",
      "target": "2RPE77ACER"
    },
    {
      "field": "uses",
      "target": "0GT4WWGWH2"
    },
    {
      "field": "uses",
      "target": "89P5GTKW87"
    },
    {
      "field": "uses",
      "target": "0KX66CZMBB"
    },
    {
      "field": "uses",
      "target": "9V7YQA85PC"
    },
    {
      "field": "uses",
      "target": "2RPE77ACER"
    },
    {
      "field": "uses",
      "target": "B8FVHNXAN1"
    }
  ],
  "markdown": "https://tamnd.github.io/math2/node/G0EDYEDG4S.md",
  "msc": [
    "65F10"
  ],
  "name": "Gauss–Seidel method",
  "prov": null,
  "role": "theorem",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/G0E/G0EDYEDG4S.md",
  "status": null,
  "tag": "G0EDYEDG4S",
  "uri": "m2://G0EDYEDG4S",
  "url": "https://tamnd.github.io/math2/wiki/gauss-seidel-method/",
  "writing": {
    "tag": "G0EDYEDG4S",
    "kind": "statement",
    "level": "W0",
    "next": [
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      }
    ]
  }
}