{
  "aliases": null,
  "backlinks": null,
  "body": "\n## Definition\n\nLet $X$ be a real $n \\times p$ matrix, let $y \\in \\R^n$, and let $\\lambda_1 \\ge 0$ and $\\lambda_2 \\ge 0$.\nFor $b \\in \\R^p$ write $\\norm{b}_1 = \\sum_{j=1}^p \\abs{b_j}$ and let $\\norm{\\cdot}_2$ be the Euclidean norm.\nAn *elastic net estimate* with parameters $\\lambda_1, \\lambda_2$ is a minimiser over $b \\in \\R^p$ of\n$$\\norm{y - X b}_2^2 + \\lambda_1 \\norm{b}_1 + \\lambda_2 \\norm{b}_2^2.$$\n\n## Remarks\n\nIf $\\lambda_2 \u003e 0$, the objective is strictly [[convex|07Q7CNM5WF]] and tends to infinity with $\\norm{b}_2$, so the estimate exists and is unique.\nWith $\\lambda_2 = 0$ it is a [[lasso|KZ226116HW]] estimate, and with $\\lambda_1 = 0$ it is the [[ridge regression|3KZW2930V2]] estimate.\nA common parametrisation uses one level $\\lambda$ and a mixing weight $\\alpha \\in [0, 1]$, with penalty $\\lambda \\big( \\alpha \\norm{b}_1 + \\frac{1 - \\alpha}{2} \\norm{b}_2^2 \\big)$.\nSome authors multiply the estimate defined above by $1 + \\lambda_2$ to undo part of the double shrinkage.\n\nExample: assume $X^T X = I_p$ and put $z = X^T y$.\nThen $\\norm{y - Xb}_2^2 = \\norm{z - b}_2^2 + \\text{const}$, and the problem splits into coordinates.\nMinimising $(z_j - b_j)^2 + \\lambda_1 \\abs{b_j} + \\lambda_2 b_j^2$ gives\n$$\\hat\\beta_j = \\frac{\\sgn(z_j) \\max(\\abs{z_j} - \\lambda_1/2, \\, 0)}{1 + \\lambda_2}.$$\nSo coefficients with $\\abs{z_j} \\le \\lambda_1 / 2$ are exactly zero, and the others are shrunk by both penalties.\n\nNon-example: if $\\lambda_1 = 0$ and $\\lambda_2 \u003e 0$, no coefficient is set to zero unless $z_j = 0$, so no selection takes place; that is ridge regression.\n",
  "commit": "ff8a40749dc16545e3d2fc637f6b0196e81d0bfa",
  "front": {
    "defines": "TKQPE0Q05Y",
    "kind": "definition",
    "msc": [
      "62J07",
      "62J05"
    ],
    "name": "Elastic net",
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/Elastic_net_regularization"
      }
    ],
    "tag": "Q8MTV1Z4AQ",
    "uses": [
      "KZ226116HW",
      "3KZW2930V2",
      "07Q7CNM5WF"
    ]
  },
  "kind": "definition",
  "link": "https://tamnd.github.io/math2/node/Q8MTV1Z4AQ/",
  "links": [
    {
      "field": "defines",
      "target": "TKQPE0Q05Y"
    },
    {
      "field": "uses",
      "target": "KZ226116HW"
    },
    {
      "field": "uses",
      "target": "3KZW2930V2"
    },
    {
      "field": "uses",
      "target": "07Q7CNM5WF"
    }
  ],
  "markdown": "https://tamnd.github.io/math2/node/Q8MTV1Z4AQ.md",
  "msc": [
    "62J07",
    "62J05"
  ],
  "name": "Elastic net",
  "prov": null,
  "role": "",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/Q8M/Q8MTV1Z4AQ.md",
  "status": null,
  "tag": "Q8MTV1Z4AQ",
  "uri": "m2://Q8MTV1Z4AQ",
  "url": "https://tamnd.github.io/math2/wiki/elastic-net-definition/",
  "writing": {
    "tag": "Q8MTV1Z4AQ",
    "kind": "definition",
    "level": "W0",
    "next": [
      {
        "what": "an inventory row",
        "note": "the row of the inventory that the node accounts for",
        "have": 0,
        "want": 1
      },
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      }
    ]
  }
}