{
  "aliases": null,
  "backlinks": null,
  "body": "\n## Definition\n\nLet $h \\colon \\R^d \\to \\R^d$ be an unknown function.\nAssume that at each point $\\theta \\in \\R^d$ one can observe a [[random variable|0DKRR22AEQ]] whose mean is $h(\\theta)$.\nA *stochastic approximation algorithm* is a recursion\n$$\\theta_{n+1} = \\theta_n + a_n Y_{n+1}, \\qquad n \\ge 0,$$\nin which:\n\n1. $\\theta_0 \\in \\R^d$ is an initial point, possibly random.\n2. $(a_n)_{n \\ge 0}$ is a [[sequence|0QE8STGJH2]] of positive step sizes.\n3. $Y_{n+1}$ is a noisy observation at $\\theta_n$: with $\\mathcal{F}_n = \\sigma(\\theta_0, Y_1, \\dots, Y_n)$, the [[conditional expectation|0YAMJRK04E]] satisfies $E[Y_{n+1} \\mid \\mathcal{F}_n] = h(\\theta_n)$.\n\nThe aim is that $\\theta_n$ converges to a zero of $h$.\n\n## Remarks\n\nExample: let $Z_1, Z_2, \\dots$ be independent with common mean $m$, and take $h(\\theta) = m - \\theta$, $Y_{n+1} = Z_{n+1} - \\theta_n$ and $a_n = 1/(n+1)$, with $\\theta_0 = 0$.\nThen $E[Y_{n+1} \\mid \\mathcal{F}_n] = m - \\theta_n$, and by induction $\\theta_n = (Z_1 + \\cdots + Z_n)/n$, the sample mean, which tends to the zero $m$ of $h$.\n\nNon-example: the deterministic Newton iteration $\\theta_{n+1} = \\theta_n - h(\\theta_n)/h'(\\theta_n)$ is not a stochastic approximation algorithm, since it needs exact values of $h$ and $h'$ rather than noisy observations, so clause 3 fails.\n\nThe usual conditions on the step sizes are $\\sum_n a_n = \\infty$ and $\\sum_n a_n^2 \u003c \\infty$.\nThe first condition lets the iterates travel any distance, and the second makes the accumulated noise $\\sum_n a_n (Y_{n+1} - h(\\theta_n))$ converge.\nWriting the noise as a martingale difference sequence, the iterates follow, in the long run, the ordinary differential equation $\\dot\\theta = h(\\theta)$.\n",
  "commit": "ff8a40749dc16545e3d2fc637f6b0196e81d0bfa",
  "front": {
    "defines": "ZPCPW4W1Y9",
    "kind": "definition",
    "msc": [
      "62L20"
    ],
    "name": "Stochastic approximation",
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/Stochastic_approximation"
      }
    ],
    "tag": "HSSC2ZZWMY",
    "uses": [
      "0DKRR22AEQ",
      "0YAMJRK04E",
      "0QE8STGJH2"
    ]
  },
  "kind": "definition",
  "link": "https://tamnd.github.io/math2/node/HSSC2ZZWMY/",
  "links": [
    {
      "field": "defines",
      "target": "ZPCPW4W1Y9"
    },
    {
      "field": "uses",
      "target": "0DKRR22AEQ"
    },
    {
      "field": "uses",
      "target": "0YAMJRK04E"
    },
    {
      "field": "uses",
      "target": "0QE8STGJH2"
    }
  ],
  "markdown": "https://tamnd.github.io/math2/node/HSSC2ZZWMY.md",
  "msc": [
    "62L20"
  ],
  "name": "Stochastic approximation",
  "prov": null,
  "role": "",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/HSS/HSSC2ZZWMY.md",
  "status": null,
  "tag": "HSSC2ZZWMY",
  "uri": "m2://HSSC2ZZWMY",
  "url": "https://tamnd.github.io/math2/wiki/stochastic-approximation-definition/",
  "writing": {
    "tag": "HSSC2ZZWMY",
    "kind": "definition",
    "level": "W0",
    "next": [
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      }
    ]
  }
}