{
  "aliases": null,
  "backlinks": null,
  "body": "\n## Definition\n\nLet $\\Theta \\subseteq \\R^p$ be a parameter set, let $x_1, \\dots, x_n$ be known regressor values in a set $\\mathcal{X}$, and let $f \\colon \\mathcal{X} \\times \\Theta \\to \\R$ be a known function.\nAssume that $\\theta \\mapsto f(x, \\theta)$ is not affine for some $x$.\nA *nonlinear regression model* states that\n$$y_i = f(x_i, \\theta) + \\varepsilon_i, \\quad i = 1, \\dots, n,$$\nfor an unknown $\\theta \\in \\Theta$ and random errors $\\varepsilon_i$ with mean $0$.\nThe *nonlinear least squares estimate* is a minimiser over $\\theta \\in \\Theta$ of\n$$S(\\theta) = \\sum_{i=1}^n \\big( y_i - f(x_i, \\theta) \\big)^2.$$\n\n## Remarks\n\nThe estimate is computed iteratively.\nThe Gauss–Newton method replaces $f(x_i, \\theta)$ near the current value $\\theta^{(k)}$ by its first-order Taylor expansion, solves the resulting linear [[least squares|0BYTQ4X1HC]] problem, and repeats.\nIt is an approximation of [[Newton's method|07EXVNF9VP]] for minimising $S$.\nThe minimiser may not be unique, and the iteration may converge to a local minimum.\n\nExample: the exponential decay model $f(x, \\theta) = \\theta_1 e^{-\\theta_2 x}$ with $\\theta_1, \\theta_2 \u003e 0$.\nThe map $\\theta \\mapsto f(x, \\theta)$ is not affine, since $\\partial f / \\partial \\theta_1 = e^{-\\theta_2 x}$ depends on $\\theta_2$ for $x \\ne 0$.\n\nNon-example: $f(x, \\theta) = \\theta_0 + \\theta_1 x + \\theta_2 x^2$ is affine in $\\theta$ for each $x$, so this model is [[linear regression|41FVC4P5JB]], although the curve is a parabola.\n\nIf the errors are independent $N(0, \\sigma^2)$, the least squares estimate is the maximum likelihood estimate.\n",
  "commit": "ff8a40749dc16545e3d2fc637f6b0196e81d0bfa",
  "front": {
    "defines": "C7E69V030F",
    "kind": "definition",
    "msc": [
      "62J02"
    ],
    "name": "Nonlinear regression",
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/Nonlinear_regression"
      }
    ],
    "tag": "A16HP3P271",
    "uses": [
      "41FVC4P5JB",
      "0BYTQ4X1HC",
      "07EXVNF9VP"
    ]
  },
  "kind": "definition",
  "link": "https://tamnd.github.io/math2/node/A16HP3P271/",
  "links": [
    {
      "field": "defines",
      "target": "C7E69V030F"
    },
    {
      "field": "uses",
      "target": "41FVC4P5JB"
    },
    {
      "field": "uses",
      "target": "0BYTQ4X1HC"
    },
    {
      "field": "uses",
      "target": "07EXVNF9VP"
    }
  ],
  "markdown": "https://tamnd.github.io/math2/node/A16HP3P271.md",
  "msc": [
    "62J02"
  ],
  "name": "Nonlinear regression",
  "prov": null,
  "role": "",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/A16/A16HP3P271.md",
  "status": null,
  "tag": "A16HP3P271",
  "uri": "m2://A16HP3P271",
  "url": "https://tamnd.github.io/math2/wiki/nonlinear-regression-definition/",
  "writing": {
    "tag": "A16HP3P271",
    "kind": "definition",
    "level": "W0",
    "next": [
      {
        "what": "an inventory row",
        "note": "the row of the inventory that the node accounts for",
        "have": 0,
        "want": 1
      },
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      }
    ]
  }
}