{
  "aliases": null,
  "backlinks": null,
  "body": "\n## Definition\n\nLet $x_1, \\dots, x_n \\in \\R^p$ be known regressor vectors.\nThe *logistic regression model* for responses $y_1, \\dots, y_n \\in \\set{0, 1}$ states three things.\nThe $y_i$ are independent, $y_i$ has the [[Bernoulli distribution|0386Y0Z7Y7]] with parameter $p_i$, and\n$$\\log \\frac{p_i}{1 - p_i} = x_i^T \\beta$$\nfor an unknown $\\beta \\in \\R^p$.\nEquivalently, $p_i = \\sigma(x_i^T \\beta)$, where $\\sigma(t) = 1/(1 + e^{-t})$ is the logistic function.\n\n## Remarks\n\nIt is the [[generalized linear model|K6C48KMRKC]] with Bernoulli responses and the logit link $g(\\mu) = \\log(\\mu / (1 - \\mu))$, which is the canonical link.\nThe log-likelihood is $\\ell(\\beta) = \\sum_i \\big( y_i x_i^T \\beta - \\log(1 + e^{x_i^T \\beta}) \\big)$, a concave function.\nThe [[maximum likelihood|0FREPFKNS7]] estimate solves $\\sum_i (y_i - p_i) x_i = 0$ and has no closed formula in general.\nFor a regressor $x_{ij}$, the number $e^{\\beta_j}$ is the odds ratio for a unit increase of $x_{ij}$ with the other regressors fixed.\n\nExample: with $x_i = (1, t_i)$ and $\\beta = (-2, 0.5)$, at $t = 4$ the log-odds is $0$, so $p = \\sigma(0) = 1/2$; at $t = 6$ it is $1$, so $p = e/(1 + e) \\approx 0.73$.\n\nNon-example: suppose the data are perfectly separated, that is, some $\\gamma \\in \\R^p$ has $x_i^T \\gamma \u003e 0$ whenever $y_i = 1$ and $x_i^T \\gamma \u003c 0$ whenever $y_i = 0$.\nThen $\\ell(c\\gamma)$ increases to $0$ as $c \\to \\infty$, while $\\ell \u003c 0$ everywhere, so the maximum likelihood estimate does not exist.\nThe model is still defined, but this fitting method fails.\n",
  "commit": "ff8a40749dc16545e3d2fc637f6b0196e81d0bfa",
  "front": {
    "defines": "JN1PJ5AJJS",
    "kind": "definition",
    "msc": [
      "62J12"
    ],
    "name": "Logistic regression",
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/Logistic_regression"
      }
    ],
    "tag": "AYTEC1KET8",
    "uses": [
      "K6C48KMRKC",
      "0386Y0Z7Y7",
      "0FREPFKNS7"
    ]
  },
  "kind": "definition",
  "link": "https://tamnd.github.io/math2/node/AYTEC1KET8/",
  "links": [
    {
      "field": "defines",
      "target": "JN1PJ5AJJS"
    },
    {
      "field": "uses",
      "target": "K6C48KMRKC"
    },
    {
      "field": "uses",
      "target": "0386Y0Z7Y7"
    },
    {
      "field": "uses",
      "target": "0FREPFKNS7"
    }
  ],
  "markdown": "https://tamnd.github.io/math2/node/AYTEC1KET8.md",
  "msc": [
    "62J12"
  ],
  "name": "Logistic regression",
  "prov": null,
  "role": "",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/AYT/AYTEC1KET8.md",
  "status": null,
  "tag": "AYTEC1KET8",
  "uri": "m2://AYTEC1KET8",
  "url": "https://tamnd.github.io/math2/wiki/logistic-regression-definition/",
  "writing": {
    "tag": "AYTEC1KET8",
    "kind": "definition",
    "level": "W0",
    "next": [
      {
        "what": "an inventory row",
        "note": "the row of the inventory that the node accounts for",
        "have": 0,
        "want": 1
      },
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      }
    ]
  }
}