---
tag: 5D0J26E619
kind: concept
name: Differential dynamic programming
msc: [49M15, 49L20, 93C55]
sources:
  - url: https://en.wikipedia.org/wiki/Differential_dynamic_programming
same_as:
  wikidata: Q5275342
  wikipedia: Differential dynamic programming
---

## Summary

Differential dynamic programming is an iterative method for discrete-time [[optimal control|T3V6ZNAMDN]] problems.
Around the current trajectory it builds second-order models of the cost-to-go and solves them backward in time, as in dynamic programming.
This gives a correction of the control together with a local linear feedback, which is applied in a forward simulation to get the next trajectory.
Near a solution it converges quadratically, like [[Newton's method|07EXVNF9VP]], and it is widely used in robotics.

## Explanations

### L1 human

Dynamic programming would compute the best decision for every possible state, which is far too much work for a robot with many joints.
Differential dynamic programming only looks near the current guess of the motion.
There it replaces the costs by quadratic approximations, which can be solved exactly, and so learns how to improve each step and how to react to small deviations.
For a linear system with quadratic costs, the approximations are exact and one iteration gives the optimal controller.
For nonlinear problems, a few iterations usually suffice.