{
  "aliases": null,
  "backlinks": null,
  "body": "\n## Definition\n\nLet $\\delta \u003e 0$.\nThe *Huber loss* with threshold $\\delta$ is the function $L_\\delta \\colon \\R \\to \\R$ given by\n$$L_\\delta(a) = \\begin{cases} \\dfrac{a^2}{2} \u0026 \\text{if } \\abs{a} \\le \\delta, \\\\[1ex] \\delta\\Big(\\abs{a} - \\dfrac{\\delta}{2}\\Big) \u0026 \\text{if } \\abs{a} \u003e \\delta. \\end{cases}$$\n\n## Remarks\n\nThe two pieces agree at $\\abs{a} = \\delta$, where both equal $\\delta^2/2$, and so do their derivatives.\nSo $L_\\delta$ is [[convex|0VV0HSW1ZS]] and continuously differentiable, with derivative $\\psi_\\delta(a) = \\max(-\\delta, \\min(\\delta, a))$, the residual clipped to $[-\\delta, \\delta]$.\nHuber's [[M-estimator|4503R8BVF8]] of location minimises $\\sum_{i=1}^{n} L_\\delta(x_i - \\mu)$ over $\\mu \\in \\R$, or equivalently solves $\\sum_i \\psi_\\delta(x_i - \\mu) = 0$.\nAs $\\delta \\to \\infty$ it tends to the mean, and as $\\delta \\to 0$ it tends to the median.\n\nExample: with $\\delta = 1$, $L_1(0.5) = 0.125$, as for squared loss, and $L_1(3) = 3 - 0.5 = 2.5$, much less than $3^2/2 = 4.5$.\n\nNon-example: the function $a \\mapsto \\abs{a}$ grows linearly but is not differentiable at $0$.\nIt is not quadratic on any interval around $0$, so it is not a Huber loss for any $\\delta$.\n",
  "commit": "ff8a40749dc16545e3d2fc637f6b0196e81d0bfa",
  "front": {
    "defines": "RATW595XR9",
    "kind": "definition",
    "msc": [
      "62F35",
      "62G35"
    ],
    "name": "Huber loss",
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/Huber_loss"
      }
    ],
    "tag": "AHYMBCMW11",
    "uses": [
      "3BTG6VVCGZ",
      "4503R8BVF8",
      "0VV0HSW1ZS"
    ]
  },
  "kind": "definition",
  "link": "https://tamnd.github.io/math2/node/AHYMBCMW11/",
  "links": [
    {
      "field": "defines",
      "target": "RATW595XR9"
    },
    {
      "field": "uses",
      "target": "3BTG6VVCGZ"
    },
    {
      "field": "uses",
      "target": "4503R8BVF8"
    },
    {
      "field": "uses",
      "target": "0VV0HSW1ZS"
    }
  ],
  "markdown": "https://tamnd.github.io/math2/node/AHYMBCMW11.md",
  "msc": [
    "62F35",
    "62G35"
  ],
  "name": "Huber loss",
  "prov": null,
  "role": "",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/AHY/AHYMBCMW11.md",
  "status": null,
  "tag": "AHYMBCMW11",
  "uri": "m2://AHYMBCMW11",
  "url": "https://tamnd.github.io/math2/wiki/huber-loss-definition/",
  "writing": {
    "tag": "AHYMBCMW11",
    "kind": "definition",
    "level": "W0",
    "next": [
      {
        "what": "an inventory row",
        "note": "the row of the inventory that the node accounts for",
        "have": 0,
        "want": 1
      },
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      }
    ]
  }
}