{
  "aliases": null,
  "backlinks": null,
  "body": "\n## Statement\n\nLet $U \\subseteq \\R^n$ be open and let $F \\colon U \\to \\R^n$ be continuously differentiable, with [[Jacobian matrix|047NDTEPCV]] $J(x)$.\nLet $x^* \\in U$ satisfy $F(x^*) = 0$, let $J(x^*)$ be [[invertible|0EB9N0S33C]], and let $J$ be [[Lipschitz continuous|0AW1E6YTJY]] near $x^*$.\n*Broyden's method* starts from $x_0 \\in U$ and an invertible $n \\times n$ matrix $B_0$, and for $k \\ge 0$ sets\n$$x_{k+1} = x_k - B_k^{-1} F(x_k), \\qquad s_k = x_{k+1} - x_k, \\qquad y_k = F(x_{k+1}) - F(x_k),$$\n$$B_{k+1} = B_k + \\frac{(y_k - B_k s_k) \\, s_k^T}{s_k^T s_k}.$$\nThen there are $\\varepsilon \u003e 0$ and $\\delta \u003e 0$ such that, if $\\norm{x_0 - x^*} \u003c \\varepsilon$ and $\\norm{B_0 - J(x^*)} \u003c \\delta$, the iterates are defined and $x_k \\to x^*$ superlinearly:\n$$\\lim_{k \\to \\infty} \\frac{\\norm{x_{k+1} - x^*}}{\\norm{x_k - x^*}} = 0,$$\nwhenever $x_k \\ne x^*$ for all $k$.\n\n## Remarks\n\nThe update makes $B_{k+1}$ satisfy the secant equation $B_{k+1} s_k = y_k$, the analogue of the [[secant method|9ZC177PVPC]] slope in several variables.\nAmong all matrices that satisfy it, $B_{k+1}$ is the closest to $B_k$ in the [[Frobenius norm|JT8Q204E4R]], and $B_{k+1} - B_k$ has rank at most $1$.\nBy the Sherman–Morrison formula, $B_{k+1}^{-1}$ is obtained from $B_k^{-1}$ in $O(n^2)$ operations, so no linear system has to be solved from scratch.\nThe proof first shows that the errors decrease linearly while $B_k$ stays close to $J(x^*)$, and then that $(B_k - J(x^*)) s_k / \\norm{s_k} \\to 0$, which is the Dennis–Moré condition for superlinear [[convergence|YWZQC5E54B]].\nThe matrices $B_k$ need not converge to $J(x^*)$.\nFor $n = 1$ the method is the secant method.\n\n## Explanations\n\n### L1 human\n\n[[Newton's method|07EXVNF9VP]] for a system of equations needs the full matrix of derivatives at every step, which can be expensive.\nBroyden's method guesses this matrix once and then corrects the guess cheaply after each step, using only the change in the function values.\nIt is slower than Newton's method near the solution, but each step is much cheaper.\n\n### L2 human\n\nBroyden's method is the first and simplest quasi-Newton method for nonlinear systems.\nIts update is the model for the BFGS and DFP updates in optimization, which keep the matrix symmetric.\nIn practice $B_0$ is often a finite-difference Jacobian, and a line search or trust region makes the method converge from farther away.\n",
  "commit": "ff8a40749dc16545e3d2fc637f6b0196e81d0bfa",
  "front": {
    "about": [
      "07EXVNF9VP"
    ],
    "kind": "statement",
    "msc": [
      "65H10"
    ],
    "name": "Broyden's method",
    "role": "theorem",
    "same_as": {
      "wikidata": "Q761993",
      "wikipedia": "Broyden's method"
    },
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/Broyden%27s_method"
      }
    ],
    "tag": "GYNYT7KTC3",
    "uses": [
      "07EXVNF9VP",
      "047NDTEPCV",
      "0AW1E6YTJY",
      "YWZQC5E54B",
      "0EB9N0S33C",
      "JT8Q204E4R",
      "9ZC177PVPC"
    ]
  },
  "kind": "statement",
  "link": "https://tamnd.github.io/math2/node/GYNYT7KTC3/",
  "links": [
    {
      "field": "about",
      "target": "07EXVNF9VP"
    },
    {
      "field": "uses",
      "target": "07EXVNF9VP"
    },
    {
      "field": "uses",
      "target": "047NDTEPCV"
    },
    {
      "field": "uses",
      "target": "0AW1E6YTJY"
    },
    {
      "field": "uses",
      "target": "YWZQC5E54B"
    },
    {
      "field": "uses",
      "target": "0EB9N0S33C"
    },
    {
      "field": "uses",
      "target": "JT8Q204E4R"
    },
    {
      "field": "uses",
      "target": "9ZC177PVPC"
    }
  ],
  "markdown": "https://tamnd.github.io/math2/node/GYNYT7KTC3.md",
  "msc": [
    "65H10"
  ],
  "name": "Broyden's method",
  "prov": null,
  "role": "theorem",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/GYN/GYNYT7KTC3.md",
  "status": null,
  "tag": "GYNYT7KTC3",
  "uri": "m2://GYNYT7KTC3",
  "url": "https://tamnd.github.io/math2/wiki/broydens-method/",
  "writing": {
    "tag": "GYNYT7KTC3",
    "kind": "statement",
    "level": "W0",
    "next": [
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      }
    ]
  }
}