Ridge regression

Summary

Ridge regression estimates the coefficients of a linear model by least squares with a penalty on the squared Euclidean norm of the coefficients. The penalty shrinks the coefficients towards zero. It makes the estimate unique and stable even when the regressors are nearly collinear or more numerous than the observations. It is a shrinkage estimator: it accepts some bias in exchange for a smaller variance. In numerical analysis the same idea is called Tikhonov regularization.