Value function

Remarks

Under these hypotheses the trajectories from $(t, x)$ stay in a bounded set, so $V$ is finite. By definition $V(T, x) = g(x)$, and a control $u$ is optimal from $(t, x)$ exactly when $J(t, x; u) = V(t, x)$. Example: $n = m = 1$, $U = [-1, 1]$, $f(t, x, u) = u$, $L = 0$ and $g(x) = \abs{x}$. From $(t, x)$ one can move toward $0$ at speed at most $1$, so $V(t, x) = \max(\abs{x} - (T - t), 0)$. This $V$ is Lipschitz but not differentiable where $\abs{x} = T - t$ and $t < T$. Non-example: the cost $J(t, x; u_0)$ of one fixed control $u_0$ is in general not the value function. For $u_0 = 0$ in the example it is $\abs{x}$, which is larger than $V(t, x)$ when $x \ne 0$ and $t < T$. For maximization problems the infimum is replaced by a supremum. For infinite-horizon problems with discounting, $V$ depends only on the initial state.