Differential dynamic programming

Summary

Differential dynamic programming is an iterative method for discrete-time optimal control problems. Around the current trajectory it builds second-order models of the cost-to-go and solves them backward in time, as in dynamic programming. This gives a correction of the control together with a local linear feedback, which is applied in a forward simulation to get the next trajectory. Near a solution it converges quadratically, like Newton's method, and it is widely used in robotics.