{
  "aliases": null,
  "backlinks": [
    {
      "field": "about",
      "target": "Y1G3NNJSQH"
    },
    {
      "field": "uses",
      "target": "Y1G3NNJSQH"
    }
  ],
  "body": "\n## Statement\n\nLet $A$ be a real $n \\times n$ matrix with nonzero diagonal entries, and let $b \\in \\R^n$.\nWrite $A = D + L + U$, with $D$ diagonal, $L$ strictly lower triangular and $U$ strictly upper triangular.\nLet $\\omega \\in \\R$ with $\\omega \\ne 0$.\nThe method of *successive over-relaxation* (SOR) with parameter $\\omega$ starts from any $x^{(0)} \\in \\R^n$ and sets\n$$x^{(k+1)} = (D + \\omega L)^{-1} \\left( \\omega b - \\left( \\omega U + (\\omega - 1) D \\right) x^{(k)} \\right).$$\nLet $G_\\omega = (D + \\omega L)^{-1} \\left( (1 - \\omega) D - \\omega U \\right)$ be its iteration matrix.\n\n1. (Kahan) The [[spectral radius|9V7YQA85PC]] satisfies $\\rho(G_\\omega) \\ge \\abs{\\omega - 1}$. So if SOR converges for every $b$ and every $x^{(0)}$, then $0 \u003c \\omega \u003c 2$.\n2. (Ostrowski–Reich) If $A$ is symmetric and [[positive definite|0KX66CZMBB]] and $0 \u003c \\omega \u003c 2$, then $\\rho(G_\\omega) \u003c 1$. Then for every $b$ and every $x^{(0)}$ the iterates converge to $A^{-1} b$.\n\n## Remarks\n\nComponentwise, SOR computes the [[Gauss–Seidel|G0EDYEDG4S]] value $\\tilde{x}_i$ for the $i$-th unknown and takes $x^{(k+1)}_i = (1 - \\omega) x^{(k)}_i + \\omega \\tilde{x}_i$.\nFor $\\omega = 1$ it is the Gauss–Seidel method.\nKahan's bound comes from the determinant: $\\det G_\\omega = \\det(D)^{-1} \\det((1 - \\omega) D) = (1 - \\omega)^n$, and this is the product of the eigenvalues.\nThe Ostrowski–Reich theorem follows because the energy norm of the error decreases strictly at each step when $A$ is positive definite and $0 \u003c \\omega \u003c 2$.\nAssume moreover that $A$ is consistently ordered, for example tridiagonal.\nAssume also that the [[Jacobi|B8FVHNXAN1]] iteration matrix has real eigenvalues and spectral radius $\\mu \u003c 1$.\nThen Young's theory gives the best parameter $\\omega_{\\mathrm{opt}} = 2 / (1 + \\sqrt{1 - \\mu^2})$, with $\\rho(G_{\\omega_{\\mathrm{opt}}}) = \\omega_{\\mathrm{opt}} - 1$.\nFor the model Poisson problem with mesh width $h$ this improves the convergence factor from $1 - O(h^2)$ to $1 - O(h)$.\n\n## Explanations\n\n### L1 human\n\nSOR takes the step that Gauss–Seidel suggests and stretches it by a factor $\\omega$.\nWhen the iterates creep slowly towards the solution, going a bit further than suggested, with $\\omega$ between $1$ and $2$, saves many steps.\nThe parameter must stay between $0$ and $2$; outside that range it cannot converge for every starting guess.\n\n### L2 human\n\nSOR was for decades the main [[iterative method|2RPE77ACER]] for large elliptic problems, because a good $\\omega$ makes it far faster than Jacobi or Gauss–Seidel.\nThe best $\\omega$ depends on the spectrum of the Jacobi matrix, which is known exactly for model problems and estimated otherwise.\nSymmetric SOR runs a forward and a backward sweep and gives a symmetric preconditioner for the conjugate gradient method.\n",
  "commit": "ff8a40749dc16545e3d2fc637f6b0196e81d0bfa",
  "front": {
    "about": [
      "2RPE77ACER"
    ],
    "kind": "statement",
    "msc": [
      "65F10"
    ],
    "name": "Successive over-relaxation",
    "role": "theorem",
    "same_as": {
      "wikidata": "Q1561270",
      "wikipedia": "Successive over-relaxation"
    },
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/Successive_over-relaxation"
      }
    ],
    "tag": "6KQJR85ZC6",
    "uses": [
      "0GT4WWGWH2",
      "0KX66CZMBB",
      "9V7YQA85PC",
      "2RPE77ACER",
      "G0EDYEDG4S",
      "B8FVHNXAN1"
    ]
  },
  "kind": "statement",
  "link": "https://tamnd.github.io/math2/node/6KQJR85ZC6/",
  "links": [
    {
      "field": "about",
      "target": "2RPE77ACER"
    },
    {
      "field": "uses",
      "target": "0GT4WWGWH2"
    },
    {
      "field": "uses",
      "target": "0KX66CZMBB"
    },
    {
      "field": "uses",
      "target": "9V7YQA85PC"
    },
    {
      "field": "uses",
      "target": "2RPE77ACER"
    },
    {
      "field": "uses",
      "target": "G0EDYEDG4S"
    },
    {
      "field": "uses",
      "target": "B8FVHNXAN1"
    }
  ],
  "markdown": "https://tamnd.github.io/math2/node/6KQJR85ZC6.md",
  "msc": [
    "65F10"
  ],
  "name": "Successive over-relaxation",
  "prov": null,
  "role": "theorem",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/6KQ/6KQJR85ZC6.md",
  "status": null,
  "tag": "6KQJR85ZC6",
  "uri": "m2://6KQJR85ZC6",
  "url": "https://tamnd.github.io/math2/wiki/successive-over-relaxation/",
  "writing": {
    "tag": "6KQJR85ZC6",
    "kind": "statement",
    "level": "W0",
    "next": [
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      }
    ]
  }
}