Role of verification theorems in stochastic optimal control? I am currently working on the optimal control of certain classes of stochastic processes and I have difficulties understanding the roles of verification theorems.
My problem is the following: I am not sure to understand whether this is purely a problem that arises from the use of partial differential equations for which we may need to consider viscosity solutions or whether this is something related to the connection between the optimal control problem and its solution expressed in terms of the value function which is a solution to the Hamilton-Jacobi-Bellman (HJB) equation, which is a PDE in many instances of those problems. Would not be the Bellman optimality principle ensures that the optimal control can be computed from the value function itself a solution of the HJB equation?
I am also wondering whether this a specificity of stochastic optimal control problems because I am not sure to have seen verification theorems in the deterministic setting.
Thanks and feel free to comment to ask for more details.
 A: On the role of verification theorem: it is an issue related to the existence-uniqueness of solutions in the classical sense for the HJB PDE. In applying the verification theorem, we ignore such issues, guess the structure of a smooth value function, formally verify (by substitution) that the guessed structural form satisfies the HJB PDE under consideration, and then use the Bellman's principle of optimality to compute the optimal control. Whether such verification is valid remains contingent on the existence-uniqueness of smooth enough classical solution (at least $C^1$ in the deterministic case and $C^2$ in the stochastic case) for the HJB PDE.
On deterministic versus stochastic: The above verification/HJB classical solution issue is for both the deterministic and the stochastic case. For example, see Ch. 4, Sec. 2 of [1], which specifically talks about verification theorems for the first order HJB PDEs in deterministic optimal control. Example 2.3 there is about an 1D deterministic optimal control problem whose HJB PDE does not admit any $C^{1}([0,T],\mathbb{R})$ solution. Ch. 4 and Ch. 5 of that book discusses details on the verification theorems for both the deterministic and stochastic optimal control problems, and also the viscosity solutions.
[1] J. Yong and X.Y. Zhou, Stochastic Controls: Hamiltonian Systems and HJB Equations, vol 43. Springer, New York, 1999.
