Ordinary least squares

Summary

Ordinary least squares estimates the coefficients of a linear regression by minimising the sum of squared residuals. The estimate has the closed form $\hat\beta = (X^T X)^{-1} X^T y$ when the design matrix has full column rank. Geometrically, the fitted values are the orthogonal projection of the data onto the column space of the design matrix. By the Gauss–Markov theorem, it is the best linear unbiased estimator under the classical assumptions.