{
  "aliases": null,
  "backlinks": null,
  "body": "\n## Definition\n\nLet $X$ be a real $n \\times p$ matrix with rows $x_1^T, \\dots, x_n^T$ and rank $p$, let $y \\in \\R^n$, and let $w_1, \\dots, w_n \u003e 0$ be weights.\nLet $W$ be the diagonal matrix with diagonal entries $w_1, \\dots, w_n$.\nThe *weighted least squares estimator* is the minimiser of\n$$\\sum_{i=1}^n w_i (y_i - x_i^T b)^2$$\nover $b \\in \\R^p$, which is\n$$\\hat\\beta_{\\mathrm{WLS}} = (X^T W X)^{-1} X^T W y.$$\n\n## Remarks\n\nExample: take $p = 1$ and $X$ the column of ones, so that $x_i^T b = b$.\nThe function $\\sum_i w_i (y_i - b)^2$ has derivative $-2 \\sum_i w_i (y_i - b)$, which vanishes at $b = \\sum_i w_i y_i / \\sum_i w_i$, the weighted mean.\nThe formula agrees: $X^T W X = \\sum_i w_i$ and $X^T W y = \\sum_i w_i y_i$.\n\nNon-example: if some weight is negative, the objective need not be bounded below.\nWith $n = 1$, $p = 1$, $x_1 = 1$ and $w_1 = -1$, the function $-(y_1 - b)^2$ has no minimum.\n\nIn a model $y = X\\beta + \\varepsilon$ with uncorrelated errors and $\\operatorname{Var}(\\varepsilon_i) = \\sigma_i^2$, the choice $w_i = 1/\\sigma_i^2$ gives the [[generalized least squares|P142WXJCF3]] estimator with $\\Omega = \\operatorname{diag}(\\sigma_1^2, \\dots, \\sigma_n^2)$.\nThis is the usual remedy for known [[heteroscedasticity|D8VX6XQ3TT]].\nWith all weights equal it is [[ordinary least squares|YM708KHV2W]].\n",
  "commit": "ff8a40749dc16545e3d2fc637f6b0196e81d0bfa",
  "front": {
    "defines": "H5CGZHBR58",
    "kind": "definition",
    "msc": [
      "62J05"
    ],
    "name": "Weighted least squares",
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/Weighted_least_squares"
      }
    ],
    "tag": "NA5X6JVK5J",
    "uses": [
      "P142WXJCF3",
      "YM708KHV2W",
      "D8VX6XQ3TT"
    ]
  },
  "kind": "definition",
  "link": "https://tamnd.github.io/math2/node/NA5X6JVK5J/",
  "links": [
    {
      "field": "defines",
      "target": "H5CGZHBR58"
    },
    {
      "field": "uses",
      "target": "P142WXJCF3"
    },
    {
      "field": "uses",
      "target": "YM708KHV2W"
    },
    {
      "field": "uses",
      "target": "D8VX6XQ3TT"
    }
  ],
  "markdown": "https://tamnd.github.io/math2/node/NA5X6JVK5J.md",
  "msc": [
    "62J05"
  ],
  "name": "Weighted least squares",
  "prov": null,
  "role": "",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/NA5/NA5X6JVK5J.md",
  "status": null,
  "tag": "NA5X6JVK5J",
  "uri": "m2://NA5X6JVK5J",
  "url": "https://tamnd.github.io/math2/wiki/weighted-least-squares-definition/",
  "writing": {
    "tag": "NA5X6JVK5J",
    "kind": "definition",
    "level": "W0",
    "next": [
      {
        "what": "an inventory row",
        "note": "the row of the inventory that the node accounts for",
        "have": 0,
        "want": 1
      },
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      }
    ]
  }
}