{
  "aliases": null,
  "backlinks": [
    {
      "field": "about",
      "target": "2PSDHH34BJ"
    },
    {
      "field": "uses",
      "target": "2PSDHH34BJ"
    }
  ],
  "body": "\n## Statement\n\nLet $f$ and $g$ be [[probability density functions|0ZY9DX8FBF]] on $\\R^d$, and let $M \\ge 1$ be a constant with $f(x) \\le M g(x)$ for all $x$.\nLet $(Y_1, V_1), (Y_2, V_2), \\ldots$ be independent pairs, where $Y_j$ has density $g$, $V_j$ has the [[uniform distribution|0AREFQ9Z65]] on $(0, 1)$, and $Y_j$ and $V_j$ are independent.\nCall trial $j$ *accepted* if $g(Y_j) \u003e 0$ and $V_j \\le f(Y_j) / (M g(Y_j))$.\nLet $\\tau$ be the index of the first accepted trial and $X = Y_\\tau$.\nThen:\n\n1. each trial is accepted with probability $1/M$;\n2. the [[conditional distribution|0990W5PN13]] of $Y_j$ given that trial $j$ is accepted has density $f$;\n3. $\\tau$ is finite almost surely, has the [[geometric distribution|04P2SR252N]] with success probability $1/M$, so $\\mathbb E\\tau = M$, and $X$ has density $f$.\n\n## Remarks\n\nFor a measurable set $A$, $P(Y_j \\in A,\\ \\text{accepted}) = \\int_A g(y) \\frac{f(y)}{M g(y)}\\, dy = \\frac{1}{M} \\int_A f(y)\\, dy$, where the integral runs over the points with $g \u003e 0$, which carry all the mass of $f$ since $f \\le M g$.\nTaking $A = \\R^d$ gives clause 1, and dividing by $1/M$ gives clause 2.\nClause 3 follows because the trials are independent and $X$ is the value of the first success.\nExample: to sample the density $f(x) = 2x$ on $[0, 1]$, take $g = 1$ on $[0, 1]$ and $M = 2$; a uniform $Y$ is accepted when $V \\le Y$, which happens half the time.\nThe bound $f \\le M g$ is necessary: if $f \u003e M g$ on a set of positive measure, the accepted values follow the density proportional to $\\min(f, M g)$, not $f$.\nThe method needs $f$ only up to a constant factor: if $f = c \\tilde f$ with unknown $c$, one uses $\\tilde f \\le \\tilde M g$ and the acceptance test $V \\le \\tilde f(Y)/(\\tilde M g(Y))$.\n\n## Explanations\n\n### L1 human\n\nTo sample from a difficult distribution, sample from an easier one that sits above it after scaling, and throw away some of the samples.\nPicture the graph of $f$ under the graph of $M g$: pick a random point under $M g$, and keep it only if it also lies under $f$.\nThe kept points are uniform under the graph of $f$, so their horizontal positions have density $f$.\nOn average $M$ tries are needed per kept sample, so the envelope $M g$ should fit $f$ closely.\n\n### L2 human\n\nRejection sampling, due to von Neumann, is exact, needs no inverse distribution function, and works in any dimension.\nIts weakness is the acceptance rate $1/M$, which typically decays exponentially with the dimension, so in high dimension Markov chain Monte Carlo is used instead.\nMany fast samplers are refinements of it.\nThe ziggurat algorithm uses a cheap envelope made of rectangles.\nAdaptive rejection sampling builds the envelope from tangent lines of $\\log f$ when $f$ is log-concave.\nThe number of trials before acceptance is geometric, so the cost per sample is random but with all moments finite.\n",
  "commit": "ff8a40749dc16545e3d2fc637f6b0196e81d0bfa",
  "front": {
    "about": [
      "0ZY9DX8FBF"
    ],
    "kind": "statement",
    "msc": [
      "65C10",
      "65C05"
    ],
    "name": "Rejection sampling",
    "role": "theorem",
    "same_as": {
      "wikidata": "Q381699",
      "wikipedia": "Rejection sampling"
    },
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/Rejection_sampling"
      }
    ],
    "tag": "R1QBNFTHFP",
    "uses": [
      "0ZY9DX8FBF",
      "0AREFQ9Z65",
      "0990W5PN13",
      "04P2SR252N"
    ]
  },
  "kind": "statement",
  "link": "https://tamnd.github.io/math2/node/R1QBNFTHFP/",
  "links": [
    {
      "field": "about",
      "target": "0ZY9DX8FBF"
    },
    {
      "field": "uses",
      "target": "0ZY9DX8FBF"
    },
    {
      "field": "uses",
      "target": "0AREFQ9Z65"
    },
    {
      "field": "uses",
      "target": "0990W5PN13"
    },
    {
      "field": "uses",
      "target": "04P2SR252N"
    }
  ],
  "markdown": "https://tamnd.github.io/math2/node/R1QBNFTHFP.md",
  "msc": [
    "65C10",
    "65C05"
  ],
  "name": "Rejection sampling",
  "prov": null,
  "role": "theorem",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/R1Q/R1QBNFTHFP.md",
  "status": null,
  "tag": "R1QBNFTHFP",
  "uri": "m2://R1QBNFTHFP",
  "url": "https://tamnd.github.io/math2/wiki/rejection-sampling/",
  "writing": {
    "tag": "R1QBNFTHFP",
    "kind": "statement",
    "level": "W0",
    "next": [
      {
        "what": "an inventory row",
        "note": "the row of the inventory that the node accounts for",
        "have": 0,
        "want": 1
      },
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      }
    ]
  }
}