{
  "aliases": null,
  "backlinks": null,
  "body": "\n## Problem\n\nLet $A \\in \\R^{m \\times n}$ and $b \\in \\R^m$.\nThe *non-negative least squares problem* (NNLS) is to find\n$$x^* \\in \\arg\\min \\set{ \\norm{Ax - b}_2^2 : x \\in \\R^n, \\ x \\ge 0 },$$\nwhere $x \\ge 0$ means $x_i \\ge 0$ for every $i$.\n\n## Remarks\n\nThe problem always has a solution.\nThe image of the non-negative orthant under $A$ is a closed convex cone, and $Ax^*$ is the nearest point of that cone to $b$.\nThe objective is a [[convex function|07Q7CNM5WF]], so NNLS is a convex quadratic programming problem, and the solution is unique when $A$ has full column rank.\nBy the Karush–Kuhn–Tucker conditions, $x$ is a solution if and only if, with $w = A^T(b - Ax)$, one has $x \\ge 0$, $w \\le 0$ and $x_i w_i = 0$ for every $i$.\nExample: for $A = I$ the solution is $x^*_i = \\max(b_i, 0)$, which satisfies these conditions.\nThe classical algorithm is the active-set method of Lawson and Hanson, published in their 1974 book *Solving Least Squares Problems*.\nIt moves variables between a free set and a set fixed at zero.\nAt each step it solves an unconstrained least squares problem on the free set, and it stops after finitely many steps.\nProjected gradient and coordinate descent methods are used for large problems.\n\n## Explanations\n\n### L1 human\n\nOrdinary least squares finds the best fit of $Ax$ to $b$ with no restriction on $x$.\nIn many applications, such as concentrations, intensities or counts, the unknowns cannot be negative, and the unrestricted fit may give negative values.\nNNLS asks for the best fit among non-negative $x$ only.\nThe answer is the closest point to $b$ in the cone of all non-negative combinations of the columns of $A$.\n",
  "commit": "ff8a40749dc16545e3d2fc637f6b0196e81d0bfa",
  "front": {
    "about": [
      "0BYTQ4X1HC"
    ],
    "history": {
      "first_stated": "1974",
      "notes": "Lawson and Hanson gave the standard active-set algorithm in Solving Least Squares Problems (1974).",
      "posed_by": "Charles L. Lawson and Richard J. Hanson"
    },
    "kind": "problem",
    "msc": [
      "65F20",
      "90C20",
      "65K05"
    ],
    "name": "Non-negative least squares",
    "same_as": {
      "wikidata": "Q17086437",
      "wikipedia": "Non-negative least squares"
    },
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/Non-negative_least_squares"
      }
    ],
    "tag": "1JV7093B4C",
    "uses": [
      "0BYTQ4X1HC",
      "07Q7CNM5WF"
    ]
  },
  "kind": "problem",
  "link": "https://tamnd.github.io/math2/node/1JV7093B4C/",
  "links": [
    {
      "field": "about",
      "target": "0BYTQ4X1HC"
    },
    {
      "field": "uses",
      "target": "0BYTQ4X1HC"
    },
    {
      "field": "uses",
      "target": "07Q7CNM5WF"
    }
  ],
  "markdown": "https://tamnd.github.io/math2/node/1JV7093B4C.md",
  "msc": [
    "65F20",
    "90C20",
    "65K05"
  ],
  "name": "Non-negative least squares",
  "prov": null,
  "role": "",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/1JV/1JV7093B4C.md",
  "status": null,
  "tag": "1JV7093B4C",
  "uri": "m2://1JV7093B4C",
  "url": "https://tamnd.github.io/math2/wiki/non-negative-least-squares/",
  "writing": {
    "tag": "1JV7093B4C",
    "kind": "problem",
    "level": "W0",
    "next": [
      {
        "what": "an inventory row",
        "note": "the row of the inventory that the node accounts for",
        "have": 0,
        "want": 1
      },
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      }
    ]
  }
}