{
  "aliases": null,
  "backlinks": [
    {
      "field": "defines",
      "target": "KCVKGSYJR9"
    },
    {
      "field": "uses",
      "target": "Q8MTV1Z4AQ"
    }
  ],
  "body": "\n## Summary\n\nRidge regression estimates the coefficients of a linear model by least squares with a penalty on the squared Euclidean norm of the coefficients.\nThe penalty shrinks the coefficients towards zero.\nIt makes the estimate unique and stable even when the regressors are nearly collinear or more numerous than the observations.\nIt is a [[shrinkage estimator|GPBSS1V3SS]]: it accepts some bias in exchange for a smaller variance.\nIn numerical analysis the same idea is called Tikhonov regularization.\n\n## Explanations\n\n### L1 human\n\nWhen two regressors are almost the same, [[ordinary least squares|YM708KHV2W]] can give huge coefficients of opposite signs that nearly cancel.\nRidge regression adds a cost for large coefficients, so such wild solutions are no longer worth it.\nA tuning parameter $\\lambda$ sets the strength of the cost: $\\lambda = 0$ is ordinary least squares and a very large $\\lambda$ pushes all coefficients to zero.\nWith a single regressor and no intercept, the ridge slope is $\\sum_i x_i y_i / (\\sum_i x_i^2 + \\lambda)$, a shrunk version of the least squares slope.\nUnlike the [[lasso|KZ226116HW]], ridge regression does not set coefficients exactly to zero.\n",
  "commit": "ff8a40749dc16545e3d2fc637f6b0196e81d0bfa",
  "front": {
    "kind": "concept",
    "msc": [
      "62J07",
      "62J05"
    ],
    "name": "Ridge regression",
    "same_as": {
      "wikidata": "Q106091291",
      "wikipedia": "Ridge regression"
    },
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/Ridge_regression"
      }
    ],
    "tag": "3KZW2930V2"
  },
  "kind": "concept",
  "link": "https://tamnd.github.io/math2/node/3KZW2930V2/",
  "links": null,
  "markdown": "https://tamnd.github.io/math2/node/3KZW2930V2.md",
  "msc": [
    "62J07",
    "62J05"
  ],
  "name": "Ridge regression",
  "prov": null,
  "role": "",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/3KZ/3KZW2930V2.md",
  "status": null,
  "tag": "3KZW2930V2",
  "uri": "m2://3KZW2930V2",
  "url": "https://tamnd.github.io/math2/wiki/ridge-regression/",
  "writing": {
    "tag": "3KZW2930V2",
    "kind": "concept",
    "level": "W0",
    "next": [
      {
        "what": "an inventory row",
        "note": "the row of the inventory that the node accounts for",
        "have": 0,
        "want": 1
      },
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      }
    ]
  }
}