{
  "aliases": null,
  "backlinks": null,
  "body": "\n## Definition\n\nLet $\\mathcal X$ and $\\mathcal Y$ be sets, let $L \\colon \\mathcal Y \\times \\mathcal Y \\to [0, \\infty)$ be a [[loss function|3BTG6VVCGZ]], and let $\\mathcal F$ be a class of functions $f \\colon \\mathcal X \\to \\mathcal Y$.\nLet $(x_1, y_1), \\dots, (x_n, y_n) \\in \\mathcal X \\times \\mathcal Y$ be data.\nThe *empirical risk* of $f \\in \\mathcal F$ is\n$$R_n(f) = \\frac1n \\sum_{i=1}^{n} L(f(x_i), y_i).$$\n*Empirical risk minimization* over $\\mathcal F$ selects a function $\\hat f \\in \\mathcal F$ with $R_n(\\hat f) = \\inf_{f \\in \\mathcal F} R_n(f)$, when such a function exists.\n\n## Remarks\n\nWhen the data are a random sample from a distribution $P$, $R_n(f)$ is an unbiased estimate of the [[risk|N26TSVRH2S]] $R(f) = E[L(f(X), Y)]$ for each fixed $f$.\nFor the selected $\\hat f$, however, $R_n(\\hat f)$ tends to underestimate $R(\\hat f)$, because $\\hat f$ was chosen to fit the same data.\n\nExample: with $\\mathcal X = \\R$, $\\mathcal Y = \\R$, the squared loss $L(a, y) = (a - y)^2$ and $\\mathcal F = \\set{x \\mapsto a + bx : a, b \\in \\R}$, empirical risk minimization is fitting a line by [[least squares|0BYTQ4X1HC]].\n\nNon-example: ridge regression minimises $R_n(f) + \\lambda \\norm{\\beta}^2$ with $\\lambda \u003e 0$ over linear functions $f(x) = \\beta^T x$.\nIts minimiser is in general not a minimiser of $R_n$ alone, so it is regularized empirical risk minimization, not empirical risk minimization.\n",
  "commit": "ff8a40749dc16545e3d2fc637f6b0196e81d0bfa",
  "front": {
    "defines": "56NVR0SS61",
    "kind": "definition",
    "msc": [
      "68Q32",
      "62H30",
      "68T05"
    ],
    "name": "Empirical risk minimization",
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/Empirical_risk_minimization"
      }
    ],
    "tag": "P4DFMJTD09",
    "uses": [
      "3BTG6VVCGZ",
      "N26TSVRH2S",
      "0BYTQ4X1HC"
    ]
  },
  "kind": "definition",
  "link": "https://tamnd.github.io/math2/node/P4DFMJTD09/",
  "links": [
    {
      "field": "defines",
      "target": "56NVR0SS61"
    },
    {
      "field": "uses",
      "target": "3BTG6VVCGZ"
    },
    {
      "field": "uses",
      "target": "N26TSVRH2S"
    },
    {
      "field": "uses",
      "target": "0BYTQ4X1HC"
    }
  ],
  "markdown": "https://tamnd.github.io/math2/node/P4DFMJTD09.md",
  "msc": [
    "68Q32",
    "62H30",
    "68T05"
  ],
  "name": "Empirical risk minimization",
  "prov": null,
  "role": "",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/P4D/P4DFMJTD09.md",
  "status": null,
  "tag": "P4DFMJTD09",
  "uri": "m2://P4DFMJTD09",
  "url": "https://tamnd.github.io/math2/wiki/empirical-risk-minimization-definition/",
  "writing": {
    "tag": "P4DFMJTD09",
    "kind": "definition",
    "level": "W0",
    "next": [
      {
        "what": "an inventory row",
        "note": "the row of the inventory that the node accounts for",
        "have": 0,
        "want": 1
      },
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      }
    ]
  }
}