{
  "aliases": null,
  "backlinks": null,
  "body": "\n## Definition\n\nLet $n \\ge p \\ge 1$.\nLet $X$ be a known $n \\times p$ real matrix, the *design matrix*, whose $i$-th row $x_i^T$ holds the explanatory variables of unit $i$.\nThe *linear regression model* for a response vector $y = (y_1, \\ldots, y_n)^T$ is\n$$y = X \\beta + \\varepsilon, \\qquad \\text{that is,} \\qquad y_i = x_i^T \\beta + \\varepsilon_i,$$\nwhere:\n\n1. $\\beta \\in \\R^p$ is an unknown vector of coefficients;\n2. $\\varepsilon = (\\varepsilon_1, \\ldots, \\varepsilon_n)^T$ is a random error vector with [[mean|0WQNRCNY3M]] $\\mathbb{E}\\varepsilon = 0$;\n3. the errors are uncorrelated with a common unknown [[variance|0477R9PYAE]] $\\sigma^2 \u003e 0$, that is, $\\operatorname{Cov}(\\varepsilon) = \\sigma^2 I_n$.\n\nUsually $X$ is assumed to have full column [[rank|0ZC9YTXZRY]] $p$, and often the errors are assumed normal.\n\n## Remarks\n\nExample: simple linear regression $y_i = \\beta_0 + \\beta_1 t_i + \\varepsilon_i$ is the case $p = 2$ with rows $x_i^T = (1, t_i)$.\nThe model $y_i = \\beta_0 + \\beta_1 t_i + \\beta_2 t_i^2 + \\varepsilon_i$ is also a linear regression, with rows $(1, t_i, t_i^2)$, because it is linear in $\\beta$.\nNon-example: $y_i = \\beta_0 e^{\\beta_1 t_i} + \\varepsilon_i$ is not linear in $(\\beta_0, \\beta_1)$, so it is a nonlinear regression model.\nThe conditions 2 and 3 are the Gauss–Markov conditions; when they hold and $X$ has full column rank, ordinary least squares is the best linear unbiased estimator of $\\beta$.\nWeaker forms allow correlated or heteroscedastic errors, which leads to generalised least squares.\n",
  "commit": "ff8a40749dc16545e3d2fc637f6b0196e81d0bfa",
  "front": {
    "defines": "41FVC4P5JB",
    "kind": "definition",
    "msc": [
      "62J05"
    ],
    "name": "Linear regression",
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/Linear_regression"
      }
    ],
    "tag": "PKFPHK189T",
    "uses": [
      "K3AMMSY8DD",
      "0WQNRCNY3M",
      "0477R9PYAE",
      "0KBFAWYRTE",
      "0ZC9YTXZRY"
    ]
  },
  "kind": "definition",
  "link": "https://tamnd.github.io/math2/node/PKFPHK189T/",
  "links": [
    {
      "field": "defines",
      "target": "41FVC4P5JB"
    },
    {
      "field": "uses",
      "target": "K3AMMSY8DD"
    },
    {
      "field": "uses",
      "target": "0WQNRCNY3M"
    },
    {
      "field": "uses",
      "target": "0477R9PYAE"
    },
    {
      "field": "uses",
      "target": "0KBFAWYRTE"
    },
    {
      "field": "uses",
      "target": "0ZC9YTXZRY"
    }
  ],
  "markdown": "https://tamnd.github.io/math2/node/PKFPHK189T.md",
  "msc": [
    "62J05"
  ],
  "name": "Linear regression",
  "prov": null,
  "role": "",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/PKF/PKFPHK189T.md",
  "status": null,
  "tag": "PKFPHK189T",
  "uri": "m2://PKFPHK189T",
  "url": "https://tamnd.github.io/math2/wiki/linear-regression-definition/",
  "writing": {
    "tag": "PKFPHK189T",
    "kind": "definition",
    "level": "W0",
    "next": [
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      }
    ]
  }
}