{
  "aliases": null,
  "backlinks": null,
  "body": "\n## Statement\n\nLet $U \\colon \\R^d \\to \\R$ be continuously differentiable with $Z = \\int_{\\R^d} e^{-U(q)} \\, dq \u003c \\infty$, and let $\\pi$ be the [[probability density|0ZY9DX8FBF]] $\\pi(q) = e^{-U(q)} / Z$.\nLet $H(q,p) = U(q) + \\tfrac{1}{2} \\abs{p}^2$ for $q, p \\in \\R^d$.\nFix a step size $\\varepsilon \u003e 0$ and a number of steps $L \\ge 1$.\nThe leapfrog step $\\Psi(q,p) = (q', p')$ is\n$$p_{1/2} = p - \\tfrac{\\varepsilon}{2} \\nabla U(q), \\qquad q' = q + \\varepsilon \\, p_{1/2}, \\qquad p' = p_{1/2} - \\tfrac{\\varepsilon}{2} \\nabla U(q'),$$\nwhere $\\nabla U$ is the [[gradient|0ABK9V9Y64]] of $U$, and $\\Phi = \\Psi^{L}$ is $L$ leapfrog steps.\nOne *Hamiltonian Monte Carlo* step from $q \\in \\R^d$ is as follows.\n\n1. Draw $p$ from the standard [[normal distribution|0XN3K7RSZY]] on $\\R^d$.\n2. Compute $(q^*, p^*) = \\Phi(q, p)$.\n3. With probability $\\min\\big(1, e^{H(q,p) - H(q^*, p^*)}\\big)$ move to $q^*$; otherwise stay at $q$.\n\nThen $\\pi$ is a [[stationary distribution|0KRH5157WP]] of this Markov transition: if $q$ has density $\\pi$, the new state also has density $\\pi$.\n\n## Remarks\n\nThe proof works on the joint density proportional to $e^{-H(q,p)}$ on $\\R^{2d}$, whose $q$-marginal is $\\pi$ and whose $p$-marginal is standard normal.\nLet $R(q,p) = (q,-p)$; then $F = R \\circ \\Phi$ is an [[involution|GP56NBN9RY]], since each leapfrog step is reversed by flipping the momentum, and $F$ preserves volume, since each of the three substeps is a shear.\nA Metropolis step with a deterministic, volume-preserving, involutive proposal $F$ and acceptance probability $\\min(1, e^{H - H \\circ F})$ satisfies [[detailed balance|CTGN6CK1ZS]] for $e^{-H}$, and $H \\circ R = H$.\nResampling $p$ in step 1 also leaves $e^{-H}$ invariant, so the composition does, and hence its $q$-marginal $\\pi$ is invariant.\nThe leapfrog map is a [[symplectic integrator|B4C25SXFRE]] for the [[Hamiltonian system|0EDKGA2FAK]] $\\dot q = p$, $\\dot p = -\\nabla U(q)$.\nThe exact flow would keep $H$ constant and always accept, and the accept step corrects the discretisation error.\nExample: for $U(q) = \\tfrac{1}{2} q^2$ on $\\R$, the target is standard normal, and with small $\\varepsilon$ the trajectories follow circles in the $(q,p)$ plane, so the chain moves far in each step.\nInvariance alone does not give convergence.\nFor this $U$, if $\\varepsilon L$ is half a period of the exact flow, the chain nearly alternates between $q$ and $-q$.\nFor this reason $\\varepsilon$ or $L$ is often randomised in practice.\nMore generally $p$ may be drawn from a normal law with a positive definite mass matrix $M$, with $\\tfrac{1}{2} p^T M^{-1} p$ in $H$ and $q' = q + \\varepsilon M^{-1} p_{1/2}$.\n\n## Explanations\n\n### L1 human\n\nHamiltonian Monte Carlo is a way to draw random samples from a probability distribution on a high-dimensional space.\nIt imagines a ball rolling on a landscape whose height is $U$, so that low places are likely places.\nAt each step the ball gets a random kick, rolls for a while under simulated physics, and its end point is the next proposal.\nA final accept-or-reject check makes the method exact despite the errors of the simulation.\nThe result says that, in the long run, the samples follow the intended distribution.\n\n### L2 human\n\nHMC is a [[Markov chain Monte Carlo|ZGNBR2SPZ0]] method that uses the gradient of the log-density to make long, directed moves.\nIt is a special case of the [[Metropolis–Hastings algorithm|MKHF449E22]] on the extended space of positions and momenta.\nThe two properties of the leapfrog map that make the acceptance rule simple are reversibility and volume preservation; energy conservation is only needed for a high acceptance rate.\nCompared with random-walk proposals, HMC scales much better with dimension for smooth targets, which is why it is the default sampler in several statistics packages.\nChoosing $\\varepsilon$ and $L$ is the main practical difficulty, and adaptive schemes such as the no-U-turn sampler address it.\n",
  "commit": "ff8a40749dc16545e3d2fc637f6b0196e81d0bfa",
  "front": {
    "about": [
      "ZGNBR2SPZ0"
    ],
    "kind": "statement",
    "msc": [
      "65C05",
      "65C40",
      "60J22"
    ],
    "name": "Hamiltonian Monte Carlo",
    "role": "proposition",
    "same_as": {
      "wikidata": "Q1639846",
      "wikipedia": "Hamiltonian Monte Carlo"
    },
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/Hamiltonian_Monte_Carlo"
      }
    ],
    "tag": "4AHBMJYCX1",
    "uses": [
      "ZGNBR2SPZ0",
      "0ZY9DX8FBF",
      "0XN3K7RSZY",
      "0ABK9V9Y64",
      "0KRH5157WP",
      "CTGN6CK1ZS",
      "0EDKGA2FAK",
      "B4C25SXFRE",
      "GP56NBN9RY",
      "MKHF449E22"
    ]
  },
  "kind": "statement",
  "link": "https://tamnd.github.io/math2/node/4AHBMJYCX1/",
  "links": [
    {
      "field": "about",
      "target": "ZGNBR2SPZ0"
    },
    {
      "field": "uses",
      "target": "ZGNBR2SPZ0"
    },
    {
      "field": "uses",
      "target": "0ZY9DX8FBF"
    },
    {
      "field": "uses",
      "target": "0XN3K7RSZY"
    },
    {
      "field": "uses",
      "target": "0ABK9V9Y64"
    },
    {
      "field": "uses",
      "target": "0KRH5157WP"
    },
    {
      "field": "uses",
      "target": "CTGN6CK1ZS"
    },
    {
      "field": "uses",
      "target": "0EDKGA2FAK"
    },
    {
      "field": "uses",
      "target": "B4C25SXFRE"
    },
    {
      "field": "uses",
      "target": "GP56NBN9RY"
    },
    {
      "field": "uses",
      "target": "MKHF449E22"
    }
  ],
  "markdown": "https://tamnd.github.io/math2/node/4AHBMJYCX1.md",
  "msc": [
    "65C05",
    "65C40",
    "60J22"
  ],
  "name": "Hamiltonian Monte Carlo",
  "prov": null,
  "role": "proposition",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/4AH/4AHBMJYCX1.md",
  "status": null,
  "tag": "4AHBMJYCX1",
  "uri": "m2://4AHBMJYCX1",
  "url": "https://tamnd.github.io/math2/wiki/hamiltonian-monte-carlo/",
  "writing": {
    "tag": "4AHBMJYCX1",
    "kind": "statement",
    "level": "W0",
    "next": [
      {
        "what": "an inventory row",
        "note": "the row of the inventory that the node accounts for",
        "have": 0,
        "want": 1
      },
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      }
    ]
  }
}