{
  "aliases": null,
  "backlinks": [
    {
      "field": "defines",
      "target": "KDZN92Z8JR"
    }
  ],
  "body": "\n## Summary\n\nAutomatic differentiation computes exact values of [[derivatives|0V1EJCT8G6]] of a function given as a computer program.\nIt applies the [[chain rule|0RXX2PT1MQ]] to each elementary operation of the program.\nSo its only errors are rounding errors, unlike [[numerical differentiation|NVH3F8FTYS]].\nIn forward mode it propagates directional derivatives along with the values; in reverse mode it propagates sensitivities backwards from the outputs.\nReverse mode computes the gradient of a scalar function at a small constant multiple of the cost of the function.\nThe backpropagation algorithm of machine learning is a special case of reverse mode.\n\n## Explanations\n\n### L1 human\n\nA program computes a complicated function by a long chain of simple steps: additions, multiplications, sines, exponentials.\nThe derivative of each simple step is known, and the chain rule says how to combine them.\nAutomatic differentiation lets the computer carry along the derivative next to each intermediate value.\nFor example, for $f(x) = x \\sin x$ at $x = 1$, it keeps the pairs $(1, 1)$ for $x$ and $(\\sin 1, \\cos 1)$ for $\\sin x$, and the product rule gives the pair $(\\sin 1, \\sin 1 + \\cos 1)$, whose second entry is $f'(1)$.\n",
  "commit": "ff8a40749dc16545e3d2fc637f6b0196e81d0bfa",
  "front": {
    "kind": "concept",
    "msc": [
      "65D25",
      "68W30"
    ],
    "name": "Automatic differentiation",
    "same_as": {
      "wikidata": "Q787371",
      "wikipedia": "Automatic differentiation"
    },
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/Automatic_differentiation"
      }
    ],
    "tag": "BW7FEZ8WM1"
  },
  "kind": "concept",
  "link": "https://tamnd.github.io/math2/node/BW7FEZ8WM1/",
  "links": null,
  "markdown": "https://tamnd.github.io/math2/node/BW7FEZ8WM1.md",
  "msc": [
    "65D25",
    "68W30"
  ],
  "name": "Automatic differentiation",
  "prov": null,
  "role": "",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/BW7/BW7FEZ8WM1.md",
  "status": null,
  "tag": "BW7FEZ8WM1",
  "uri": "m2://BW7FEZ8WM1",
  "url": "https://tamnd.github.io/math2/wiki/automatic-differentiation/",
  "writing": {
    "tag": "BW7FEZ8WM1",
    "kind": "concept",
    "level": "W0",
    "next": [
      {
        "what": "an inventory row",
        "note": "the row of the inventory that the node accounts for",
        "have": 0,
        "want": 1
      },
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      }
    ]
  }
}