{
  "aliases": null,
  "backlinks": [
    {
      "field": "about",
      "target": "5P6NB8P1HG"
    },
    {
      "field": "uses",
      "target": "5P6NB8P1HG"
    },
    {
      "field": "uses",
      "target": "M9SVV4FDZV"
    },
    {
      "field": "defines",
      "target": "P4DFMJTD09"
    }
  ],
  "body": "\n## Summary\n\nEmpirical risk minimization is the principle of choosing, from a class of prediction rules, one that has the smallest average [[loss|3BTG6VVCGZ]] on the training data.\nThe average loss on the data, the empirical risk, stands in for the true risk, which depends on an unknown distribution.\n[[Ordinary least squares|YM708KHV2W]] and [[maximum likelihood estimation|0FREPFKNS7]] are instances.\nWhether it works is a question of [[statistical learning theory|5MR9ZSC3CX]].\nIt succeeds when the empirical risk is uniformly close to the true risk over the class.\nIn binary classification this holds for classes of finite [[VC dimension|KCFXR9K3GD]].\n\n## Explanations\n\n### L1 human\n\nYou cannot test a rule on all future data, so you test it on the data you have and pick the rule that does best.\nFor example, to fit a line through points, choose the line with the smallest average squared vertical distance to the points.\nThis is a good idea when the family of rules is not too rich.\nIf the family can fit anything, the winner may just memorise the data and predict badly on new points.\nAdding a penalty for complexity, as in [[ridge regression|3KZW2930V2]], is the usual fix.\n",
  "commit": "ff8a40749dc16545e3d2fc637f6b0196e81d0bfa",
  "front": {
    "kind": "concept",
    "msc": [
      "68Q32",
      "62H30",
      "68T05"
    ],
    "name": "Empirical risk minimization",
    "same_as": {
      "wikidata": "Q5374254",
      "wikipedia": "Empirical risk minimization"
    },
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/Empirical_risk_minimization"
      }
    ],
    "tag": "56NVR0SS61"
  },
  "kind": "concept",
  "link": "https://tamnd.github.io/math2/node/56NVR0SS61/",
  "links": null,
  "markdown": "https://tamnd.github.io/math2/node/56NVR0SS61.md",
  "msc": [
    "68Q32",
    "62H30",
    "68T05"
  ],
  "name": "Empirical risk minimization",
  "prov": null,
  "role": "",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/56N/56NVR0SS61.md",
  "status": null,
  "tag": "56NVR0SS61",
  "uri": "m2://56NVR0SS61",
  "url": "https://tamnd.github.io/math2/wiki/empirical-risk-minimization/",
  "writing": {
    "tag": "56NVR0SS61",
    "kind": "concept",
    "level": "W0",
    "next": [
      {
        "what": "an inventory row",
        "note": "the row of the inventory that the node accounts for",
        "have": 0,
        "want": 1
      },
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      }
    ]
  }
}