{
  "aliases": null,
  "backlinks": null,
  "body": "\n## Definition\n\nLet $H$ be a real or complex [[inner product space|0QVNCJXRY3]] with inner product $\\ip{\\cdot}{\\cdot}$ and [[induced norm|0KG4D8AP6G]] $\\norm{\\cdot}$.\nLet $V \\subseteq H$ be a finite-dimensional subspace and let $f \\in H$.\nA *least squares approximation* of $f$ from $V$ is an element $v^* \\in V$ such that\n$$\\norm{f - v^*} = \\min_{v \\in V} \\norm{f - v}.$$\nIn the main case, $w$ is a positive weight on $[a, b]$ and $H$ is the space $L^2_w$ of functions with $\\int_a^b w \\abs{f}^2 \u003c \\infty$.\nIts inner product is $\\ip{f}{g} = \\int_a^b w(x) f(x) \\overline{g(x)} \\, dx$, and $V$ is the space of polynomials of degree at most $n$.\n\n## Remarks\n\nThe least squares approximation exists, is unique, and is the [[orthogonal projection|0B8KZRT28M]] of $f$ onto $V$.\nIt is characterised by $\\ip{f - v^*}{v} = 0$ for all $v \\in V$.\nIf $\\varphi_0, \\dots, \\varphi_n$ is an orthogonal basis of $V$, then $v^* = \\sum_{k=0}^{n} \\frac{\\ip{f}{\\varphi_k}}{\\ip{\\varphi_k}{\\varphi_k}} \\varphi_k$.\nFor a general basis, the coefficients solve the normal equations, whose matrix is the Gram matrix $(\\ip{\\varphi_j}{\\varphi_k})$.\n\nExample: $f(x) = x^2$ on $[-1, 1]$ with $w = 1$ and $V$ the polynomials of degree at most $1$.\nWith the orthogonal basis $1, x$, one gets $\\ip{f}{1} / \\ip{1}{1} = (2/3)/2 = 1/3$ and $\\ip{f}{x} = 0$, so $v^* = 1/3$.\nCheck: $\\int_{-1}^{1} (x^2 - 1/3) \\, dx = 0$ and $\\int_{-1}^{1} (x^2 - 1/3) x \\, dx = 0$.\n\nNon-example: the uniform best line for $x^2$ on $[-1, 1]$ is $1/2$, not $1/3$.\nIt minimises the largest error, not the integral of the squared error, so it fails the defining condition.\n\nA discrete version replaces the integral by a weighted sum $\\sum_i w_i \\abs{f(x_i) - v(x_i)}^2$ over data points; this is curve fitting.\n",
  "commit": "ff8a40749dc16545e3d2fc637f6b0196e81d0bfa",
  "front": {
    "defines": "2QV28WYQTQ",
    "kind": "definition",
    "msc": [
      "41A10",
      "65D10",
      "41A50"
    ],
    "name": "Least squares approximation",
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/Least-squares_function_approximation"
      }
    ],
    "tag": "P3A4EAA3RA",
    "uses": [
      "0QVNCJXRY3",
      "0B8KZRT28M",
      "0KG4D8AP6G"
    ]
  },
  "kind": "definition",
  "link": "https://tamnd.github.io/math2/node/P3A4EAA3RA/",
  "links": [
    {
      "field": "defines",
      "target": "2QV28WYQTQ"
    },
    {
      "field": "uses",
      "target": "0QVNCJXRY3"
    },
    {
      "field": "uses",
      "target": "0B8KZRT28M"
    },
    {
      "field": "uses",
      "target": "0KG4D8AP6G"
    }
  ],
  "markdown": "https://tamnd.github.io/math2/node/P3A4EAA3RA.md",
  "msc": [
    "41A10",
    "65D10",
    "41A50"
  ],
  "name": "Least squares approximation",
  "prov": null,
  "role": "",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/P3A/P3A4EAA3RA.md",
  "status": null,
  "tag": "P3A4EAA3RA",
  "uri": "m2://P3A4EAA3RA",
  "url": "https://tamnd.github.io/math2/wiki/least-squares-approximation-definition/",
  "writing": {
    "tag": "P3A4EAA3RA",
    "kind": "definition",
    "level": "W0",
    "next": [
      {
        "what": "an inventory row",
        "note": "the row of the inventory that the node accounts for",
        "have": 0,
        "want": 1
      },
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      }
    ]
  }
}