Viscosity solution of the Hamilton–Jacobi–Bellman equation

Remarks

The proof uses the dynamic programming principle. If $V - \varphi$ has a local maximum at $(t_0, x_0)$, use a constant control $u$ on $[t_0, t_0 + h]$: the principle gives $\varphi(t_0, x_0) - \varphi(t_0 + h, y(t_0 + h)) \le \int_{t_0}^{t_0 + h} L \, ds$. Dividing by $h$ and letting $h \to 0$ gives $-\varphi_t - \nabla_x \varphi \cdot f(t_0, x_0, u) - L(t_0, x_0, u) \le 0$ for every $u \in U$, which is condition 1. Condition 2 uses nearly optimal controls in the same way, with the compactness of $U$. Example: for $f = u$, $U = [-1, 1]$, $L = 0$ and $g(x) = \abs{x}$, the value function $\max(\abs{x} - (T - t), 0)$ is not differentiable, yet it is a viscosity solution of $-V_t + \abs{V_x} = 0$. Here $g$ is unbounded, but one checks the two conditions directly at the corners. By the comparison principle of Crandall and Lions, $V$ is the only bounded uniformly continuous viscosity solution with $V(T, \cdot) = g$. So the equation characterizes the value function even when no classical solution exists.

Going deeper

For a student

The theorem is the bridge between optimal control and the theory of viscosity solutions. Smooth test functions $\varphi$ touching $V$ from above or below replace the missing derivatives of $V$. Combined with the comparison principle, it gives convergence proofs for numerical schemes, which only need to be monotone, stable and consistent. It also explains why the verification theorem for smooth solutions is not enough in general.