Math 2.0

Numerical analysis Concept

Least squares

Builds on Orthogonal projection is the closest pointOrthogonal projectionTranspose

Summary

Let AA be a real m×nm \times n matrix and let b∈ℝmb \in \R^m. A least squares solution of Ax=bAx = b is a vector x^∈ℝn\hat x \in \R^n that minimises ∥Ax−b∥\norm{Ax - b}, for the Euclidean norm. The vectors x^\hat x that do this are exactly the solutions of the normal equations ATAx^=ATb.A^T A \hat x = A^T b. If the columns of AA are linearly independent, then ATAA^T A is invertible, and x^=(ATA)−1ATb\hat x = (A^T A)^{-1} A^T b is unique.

Remarks

The vector Ax^A \hat x is the orthogonal projection of bb onto the column space of AA, see the closest point property. The normal equations say that the residual b−Ax^b - A\hat x is orthogonal to each column of AA. Fitting a line y=c0+c1ty = c_0 + c_1 t to data points (ti,yi)(t_i, y_i) is a least squares problem. In practice, the QR decomposition gives a more stable method than the normal equations.

Reading path

To understand this page from the ground up, read these 19 pages in order. The foundations come first.

Show the path as a graph
Matrix, definition, no evidence levelMatrixdefinitionVector space, definition, no evidence levelVector spacedefinitionTranspose, definition, no evidence levelTransposedefinitionInner product, definition, no evidence levelInner productdefinitionLinear map, definition, no evidence levelLinear mapdefinitionLinear subspace, definition, no evidence levelLinear subspacedefinitionNorm, definition, no evidence levelNormdefinitionInner product space, definition, no evidence levelInner product spacedefinitionOrthogonal vectors, definition, no evidence levelOrthogonal vectorsdefinitionEndomorphism, definition, no evidence levelEndomorphismdefinitionSum of subspaces, definition, no evidence levelSum of subspacesdefinitionNorm induced by an inner product, definition, no evidence levelNorm induced by an in…definitionOrthogonal complement, definition, no evidence levelOrthogonal complementdefinitionDirect sum of subspaces, definition, no evidence levelDirect sum of subspac…definitionA space is the direct sum of a subspace and its orthogonal complement, statement, no evidence levelA space is the direct…statementProjection, definition, no evidence levelProjectiondefinitionOrthogonal projection, definition, no evidence levelOrthogonal projectiondefinitionOrthogonal projection is the closest point, statement, no evidence levelOrthogonal projection…statementLeast squares, concept, no evidence levelLeast squaresconcept
Each page uses the pages above it. The colour of the bar on a box shows how strong the evidence is, and the box also gives it as text. The list has the same pages.
  • no evidence level
  1. Matrix definition
  2. Vector space definition
  3. Transpose definition
  4. Inner product definition
  5. Linear map definition
  6. Linear subspace definition
  7. Norm definition
  8. Inner product space definition
  9. Orthogonal vectors definition
  10. Endomorphism definition
  11. Sum of subspaces definition
  12. Norm induced by an inner product definition
  13. Orthogonal complement definition
  14. Direct sum of subspaces definition
  15. A space is the direct sum of a subspace and its orthogonal complement statement
  16. Projection definition
  17. Orthogonal projection definition
  18. Orthogonal projection is the closest point statement
  19. Least squares concept

Where this leads

19 pages build on this one.

Further reading

  1. Web https://en.wikipedia.org/wiki/Linear_least_squares, Basic formulation
About this page
Tag0BYTQ4X1HC
URIm2://0BYTQ4X1HC
Kindconcept
WritingW0 stub
MSC65F20, 15A06
UsesOrthogonal projection is the closest point, Orthogonal projection, Transpose
Same as

Markdown · JSON · Edit on GitHub

What this page needs for W1

  • an inventory row: the row of the inventory that the node accounts for (0 of 1)

The writing levels say how complete the text of a page is. They do not measure its truth.

What links here

Results about this

Used by

Special cases