Stochastic optimal control

Definition

Let $(\Omega, \mathcal{F}, (\mathcal{F}_t)_{t \ge 0}, P)$ be a probability space with a filtration that carries a $d$-dimensional Wiener process $W$. Let $T > 0$ and let $U \subseteq \R^m$ be a nonempty closed set, the control set. Let $b \colon \R^n \times U \to \R^n$ and $\sigma \colon \R^n \times U \to \R^{n \times d}$ be continuous. Assume that they are Lipschitz in $x$ uniformly in $u$, and that $\abs{b(x, u)} + \abs{\sigma(x, u)} \le K (1 + \abs{x} + \abs{u})$ for a constant $K$. Let $f \colon \R^n \times U \to \R$ and $g \colon \R^n \to \R$ be continuous with $\abs{f(x, u)} \le K (1 + \abs{x}^2 + \abs{u}^2)$ and $\abs{g(x)} \le K (1 + \abs{x}^2)$.

1. An admissible control is a progressively measurable process $u = (u_t)_{0 \le t \le T}$ with values in $U$ and $E \int_0^T \abs{u_t}^2 \, dt < \infty$.
2. For an admissible control $u$ and $x \in \R^n$, the state is the solution $X^u$ of the stochastic differential equation $dX_t = b(X_t, u_t) \, dt + \sigma(X_t, u_t) \, dW_t$ with $X_0 = x$.
3. The cost of $u$ is $J(x; u) = E\left[ \int_0^T f(X^u_t, u_t) \, dt + g(X^u_T) \right]$, an expected value.

The stochastic optimal control problem is to find an admissible control $u^*$ with $J(x; u^*) = \inf_u J(x; u)$, the infimum over all admissible controls. Such a $u^*$ is an optimal control, and $V(x) = \inf_u J(x; u)$ is the value.