{
  "aliases": null,
  "backlinks": null,
  "body": "\n## Definition\n\nConsider a finite-horizon [[optimal control|T3V6ZNAMDN]] problem on $[0, T]$ with dynamics $y' = f(s, y, u)$, controls with values in a set $U$, running cost $L$ and terminal cost $g$.\nLet $V$ be its [[value function|K6G0DRQ3CP]].\nThe *dynamic programming principle* for the problem is the identity, for all $0 \\le t \\le s \\le T$ and $x \\in \\R^n$,\n$$V(t, x) = \\inf_{u} \\Big( \\int_t^s L(r, y(r), u(r)) \\, dr + V(s, y(s)) \\Big).$$\nHere the infimum is over measurable controls $u \\colon [t, s] \\to U$, and $y$ is the trajectory with $y(t) = x$.\n*Dynamic programming in continuous time* is the method that derives from this principle a partial differential equation for $V$.\nThat equation is the [[Hamilton–Jacobi–Bellman equation|SJP9HW82VX]] $V_t + \\min_{u \\in U} \\big( L(t, x, u) + \\nabla_x V \\cdot f(t, x, u) \\big) = 0$ with $V(T, x) = g(x)$.\nOne then obtains optimal controls by choosing, at each $(t, x)$, a minimizer $u$ of this expression.\n\n## Remarks\n\nThe principle holds under the standing hypotheses of the value function: $U$ compact, $f$ continuous and Lipschitz in the state, and $L$, $g$ continuous.\nIts proof splits a control on $[t, T]$ into its parts on $[t, s]$ and $[s, T]$.\nExample: for $y' = u$, $U = [-1, 1]$, $L = 0$ and $g(x) = \\abs{x}$, the value function is $V(t, x) = \\max(\\abs{x} - (T - t), 0)$.\nWhere $\\abs{x} \u003e T - t$ it equals $\\abs{x} - (T - t)$, so $V_t = 1$ and $\\min_{\\abs{u} \\le 1} V_x u = -1$, and the equation holds.\nThe minimizer $u = -\\sgn(x)$ is the optimal feedback.\nNon-example: a greedy rule that minimizes only the running cost $L$ at each instant, ignoring $V$, does not follow the principle and is in general not optimal.\n",
  "commit": "ff8a40749dc16545e3d2fc637f6b0196e81d0bfa",
  "front": {
    "defines": "QPS8VSJGMJ",
    "kind": "definition",
    "msc": [
      "49L20",
      "49L12",
      "93E20"
    ],
    "name": "Dynamic programming in continuous time",
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/Dynamic_programming"
      }
    ],
    "tag": "F64JX05VJX",
    "uses": [
      "K6G0DRQ3CP",
      "T3V6ZNAMDN",
      "SJP9HW82VX"
    ]
  },
  "kind": "definition",
  "link": "https://tamnd.github.io/math2/node/F64JX05VJX/",
  "links": [
    {
      "field": "defines",
      "target": "QPS8VSJGMJ"
    },
    {
      "field": "uses",
      "target": "K6G0DRQ3CP"
    },
    {
      "field": "uses",
      "target": "T3V6ZNAMDN"
    },
    {
      "field": "uses",
      "target": "SJP9HW82VX"
    }
  ],
  "markdown": "https://tamnd.github.io/math2/node/F64JX05VJX.md",
  "msc": [
    "49L20",
    "49L12",
    "93E20"
  ],
  "name": "Dynamic programming in continuous time",
  "prov": null,
  "role": "",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/F64/F64JX05VJX.md",
  "status": null,
  "tag": "F64JX05VJX",
  "uri": "m2://F64JX05VJX",
  "url": "https://tamnd.github.io/math2/wiki/dynamic-programming-in-continuous-time-definition/",
  "writing": {
    "tag": "F64JX05VJX",
    "kind": "definition",
    "level": "W0",
    "next": [
      {
        "what": "an inventory row",
        "note": "the row of the inventory that the node accounts for",
        "have": 0,
        "want": 1
      },
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      }
    ]
  }
}