Classical stochastic optimal control problem is to minimize functional $$ J(u) = \mathbf E \int_0^T f(t,x_t,u_t)dt, \tag{1} $$ subject to SDE $$ dx_t = b(t,x_t,u_t)dt + \sigma(t,x_t,u_t)dW_t, \quad x_0 = \mathbf x, \tag{2} $$ where $b$ and $\sigma$ are random functions: $\mathbf R_+\times\mathbf R\times\mathbf R\times\Omega \to \mathbf R$. As far as I know there are two major approaches to such problems: maximum principle and dynamic programming.

My question is how to solve optimal control problems when instead of equation (2) we have the following equation$$ dx_t = b(t,x_t,u_t)dt, \quad x_0 = \mathbf x, \tag{3} $$ (same random drift coefficient $b$, but no diffusion $\sigma$)?. Is there special theory for this case or can we apply same methods as problem (1)-(2)?

update: To make it clear, drift is of the following form $$b=b(t,x_t,u_t,\omega):\mathbf R_+\times\mathbf R\times\mathbf R\times\Omega \to \mathbf R,$$ as particular case $b=b(t,x_t,u_t,\xi_t)$, where $\xi_t$ is some other process.

  • $\begingroup$ When the coefficients are random, the maximum principle is the best approach, since PDE methods won't work. In the case $\sigma \equiv 0$, the maximum principle (see, for example, H. Pham's book from 2009) still applies, but the adjoint equations will become more difficult. In particular, you will arrive at a forward-backward SDE (FBSDE) with a degenerate volatility, and the best bet to solve it will probably be Peng-Wu's result. If this doesn't sound familiar, I can elaborate later. $\endgroup$ – Dan Nov 2 '12 at 15:03
  • $\begingroup$ Sorry if I formulated it not very clear, I meant $b=b(t,x_t,u_t,\omega)$, as particular case $b=b(t,x_t,u_t,\xi_t)$, where $\xi_t$ is some other process. In Pham's book they consider only case where $b=b(t,x_t,u_t)$ and $b$ is deterministic, and that's not exactly what I'm looking for. I am a little bit familiar with maximum principle and FBSDEs, but never heard about Peng-Wu's result. I would appreciate if you gave me references on it. $\endgroup$ – niyazets Nov 5 '12 at 8:00
  • $\begingroup$ In this case the value function becomes an adapted process and the HJB equation a BSPDE, see Peng 1992. $\endgroup$ – user39563 Sep 4 '13 at 20:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.