{
  "aliases": null,
  "backlinks": null,
  "body": "\n## Definition\n\nLet $K \\ge 2$ be the number of classes.\nFor $k = 1, \\ldots, K$ let $\\pi_k \u003e 0$ be prior probabilities with $\\pi_1 + \\cdots + \\pi_K = 1$, and let $\\mu_k \\in \\R^p$ be class means.\nLet $\\Sigma$ be a [[positive definite|0KX66CZMBB]] $p \\times p$ matrix, the common [[covariance matrix|0V7G6WMFK0]].\nThe *linear discriminant functions* are\n$$\\delta_k(x) = x^T \\Sigma^{-1} \\mu_k - \\tfrac{1}{2} \\mu_k^T \\Sigma^{-1} \\mu_k + \\log \\pi_k, \\qquad x \\in \\R^p.$$\n*Linear discriminant analysis* assigns $x$ to a class $k$ for which $\\delta_k(x)$ is largest.\nIn practice the parameters are estimated from a labelled training sample.\nOne uses the class proportions for $\\pi_k$, the class sample means for $\\mu_k$ and the pooled sample covariance matrix for $\\Sigma$.\n\n## Remarks\n\nIf the class $k$ has the distribution $N_p(\\mu_k, \\Sigma)$ and prior probability $\\pi_k$, this rule is the Bayes rule.\nIndeed, $\\delta_k(x)$ differs from $\\log(\\pi_k f_k(x))$ by a term that does not depend on $k$, where $f_k$ is the density of class $k$.\nEach $\\delta_k$ is affine in $x$, so the boundary between two classes is a hyperplane.\nFor two classes, $x$ goes to class $1$ when\n$$(\\mu_1 - \\mu_2)^T \\Sigma^{-1} x - \\tfrac{1}{2} (\\mu_1 - \\mu_2)^T \\Sigma^{-1} (\\mu_1 + \\mu_2) \u003e \\log(\\pi_2 / \\pi_1).$$\nExample: with $p = 1$, $\\mu_1 = 1$, $\\mu_2 = -1$, $\\Sigma = 1$ and $\\pi_1 = \\pi_2$, the condition is $2x \u003e 0$, so $x$ goes to class $1$ when $x \u003e 0$.\nNon-example: if the normal classes have different covariance matrices $\\Sigma_k$, the Bayes rule has quadratic boundaries.\nThis rule is quadratic discriminant analysis, not a linear rule.\n",
  "commit": "ff8a40749dc16545e3d2fc637f6b0196e81d0bfa",
  "front": {
    "defines": "ZATS7NV9WD",
    "kind": "definition",
    "msc": [
      "62H30"
    ],
    "name": "Linear discriminant analysis",
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/Linear_discriminant_analysis"
      }
    ],
    "tag": "GTBMW9QXST",
    "uses": [
      "0XN3K7RSZY",
      "0KX66CZMBB",
      "0V7G6WMFK0"
    ]
  },
  "kind": "definition",
  "link": "https://tamnd.github.io/math2/node/GTBMW9QXST/",
  "links": [
    {
      "field": "defines",
      "target": "ZATS7NV9WD"
    },
    {
      "field": "uses",
      "target": "0XN3K7RSZY"
    },
    {
      "field": "uses",
      "target": "0KX66CZMBB"
    },
    {
      "field": "uses",
      "target": "0V7G6WMFK0"
    }
  ],
  "markdown": "https://tamnd.github.io/math2/node/GTBMW9QXST.md",
  "msc": [
    "62H30"
  ],
  "name": "Linear discriminant analysis",
  "prov": null,
  "role": "",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/GTB/GTBMW9QXST.md",
  "status": null,
  "tag": "GTBMW9QXST",
  "uri": "m2://GTBMW9QXST",
  "url": "https://tamnd.github.io/math2/wiki/linear-discriminant-analysis-definition/",
  "writing": {
    "tag": "GTBMW9QXST",
    "kind": "definition",
    "level": "W0",
    "next": [
      {
        "what": "an inventory row",
        "note": "the row of the inventory that the node accounts for",
        "have": 0,
        "want": 1
      },
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      }
    ]
  }
}