{
  "aliases": null,
  "backlinks": null,
  "body": "\n## Definition\n\nLet $X$ be a random vector in $\\R^p$ with finite second moments, mean $\\mu$ and [[covariance matrix|0V7G6WMFK0]] $\\Sigma$.\nBy the [[spectral theorem|06J4PXRG22]], $\\Sigma$ has an [[orthonormal basis|0S9GGTR0EX]] of [[eigenvectors|0N6960W0BV]] $v_1, \\ldots, v_p$ with [[eigenvalues|0BQAA2BK3Q]] $\\lambda_1 \\ge \\lambda_2 \\ge \\cdots \\ge \\lambda_p \\ge 0$.\nThe $k$-th *principal component* of $X$ is the random variable\n$$Y_k = v_k^T (X - \\mu), \\qquad k = 1, \\ldots, p.$$\nThe vector $v_k$ is the $k$-th principal direction, or loading vector.\n*Principal component analysis* is the passage from $X$ to $(Y_1, \\ldots, Y_p)$, usually followed by keeping only $Y_1, \\ldots, Y_q$ for some $q \u003c p$.\nFor a data matrix, the same construction is applied to the sample covariance matrix.\n\n## Remarks\n\nThe components are uncorrelated and $\\operatorname{Var}(Y_k) = \\lambda_k$.\nEquivalently, $v_1$ maximises the [[variance|0477R9PYAE]] $\\operatorname{Var}(a^T X) = a^T \\Sigma a$ over unit vectors $a$, and $v_k$ maximises it over unit vectors orthogonal to $v_1, \\ldots, v_{k-1}$.\nThe principal directions are unique up to sign when the eigenvalues are distinct.\n\nExample: let $\\Sigma = \\begin{pmatrix} 2 \u0026 1 \\\\ 1 \u0026 2 \\end{pmatrix}$.\nIts eigenvalues are $3$ and $1$, with unit eigenvectors $v_1 = (1, 1)/\\sqrt{2}$ and $v_2 = (1, -1)/\\sqrt{2}$.\nCheck: $v_1^T \\Sigma v_1 = (2 + 1 + 1 + 2)/2 = 3$.\nNon-example: the direction $a = (1, 0)$ gives $a^T \\Sigma a = 2 \u003c 3$, so $a^T (X - \\mu)$ is not the first principal component.\n\nThe fraction $(\\lambda_1 + \\cdots + \\lambda_q)/(\\lambda_1 + \\cdots + \\lambda_p)$ is the proportion of the total variance kept by the first $q$ components.\nFor a centred data matrix, the principal directions are the right singular vectors of its singular value decomposition.\n",
  "commit": "ff8a40749dc16545e3d2fc637f6b0196e81d0bfa",
  "front": {
    "defines": "ABXD13M340",
    "kind": "definition",
    "msc": [
      "62H25",
      "15A18"
    ],
    "name": "Principal component analysis",
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/Principal_component_analysis"
      }
    ],
    "tag": "R2EHQ06B98",
    "uses": [
      "0V7G6WMFK0",
      "0N6960W0BV",
      "0BQAA2BK3Q",
      "0S9GGTR0EX",
      "06J4PXRG22",
      "0477R9PYAE"
    ]
  },
  "kind": "definition",
  "link": "https://tamnd.github.io/math2/node/R2EHQ06B98/",
  "links": [
    {
      "field": "defines",
      "target": "ABXD13M340"
    },
    {
      "field": "uses",
      "target": "0V7G6WMFK0"
    },
    {
      "field": "uses",
      "target": "0N6960W0BV"
    },
    {
      "field": "uses",
      "target": "0BQAA2BK3Q"
    },
    {
      "field": "uses",
      "target": "0S9GGTR0EX"
    },
    {
      "field": "uses",
      "target": "06J4PXRG22"
    },
    {
      "field": "uses",
      "target": "0477R9PYAE"
    }
  ],
  "markdown": "https://tamnd.github.io/math2/node/R2EHQ06B98.md",
  "msc": [
    "62H25",
    "15A18"
  ],
  "name": "Principal component analysis",
  "prov": null,
  "role": "",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/R2E/R2EHQ06B98.md",
  "status": null,
  "tag": "R2EHQ06B98",
  "uri": "m2://R2EHQ06B98",
  "url": "https://tamnd.github.io/math2/wiki/principal-component-analysis-definition/",
  "writing": {
    "tag": "R2EHQ06B98",
    "kind": "definition",
    "level": "W0",
    "next": [
      {
        "what": "an inventory row",
        "note": "the row of the inventory that the node accounts for",
        "have": 0,
        "want": 1
      },
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      }
    ]
  }
}