MathOverflow is a question and answer site for professional mathematicians. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Classical stochastic optimal control problem is to minimize functional $$ J(u) = \mathbf E \int_0^T f(t,x_t,u_t)dt, \tag{1} $$ subject to SDE $$ dx_t = b(t,x_t,u_t)dt + \sigma(t,x_t,u_t)dW_t, \quad x_0 = \mathbf x, \tag{2} $$ where $b$ and $\sigma$ are random functions: $\mathbf R_+\times\mathbf R\times\mathbf R\times\Omega \to \mathbf R$. As far as I know there are two major approaches to such problems: maximum principle and dynamic programming.

My question is how to solve optimal control problems when instead of equation (2) we have the following equation$$ dx_t = b(t,x_t,u_t)dt, \quad x_0 = \mathbf x, \tag{3} $$ (same random drift coefficient $b$, but no diffusion $\sigma$)?. Is there special theory for this case or can we apply same methods as problem (1)-(2)?

update: To make it clear, drift is of the following form $$b=b(t,x_t,u_t,\omega):\mathbf R_+\times\mathbf R\times\mathbf R\times\Omega \to \mathbf R,$$ as particular case $b=b(t,x_t,u_t,\xi_t)$, where $\xi_t$ is some other process.

share|cite|improve this question
When the coefficients are random, the maximum principle is the best approach, since PDE methods won't work. In the case $\sigma \equiv 0$, the maximum principle (see, for example, H. Pham's book from 2009) still applies, but the adjoint equations will become more difficult. In particular, you will arrive at a forward-backward SDE (FBSDE) with a degenerate volatility, and the best bet to solve it will probably be Peng-Wu's result. If this doesn't sound familiar, I can elaborate later. – Dan Nov 2 '12 at 15:03
Sorry if I formulated it not very clear, I meant $b=b(t,x_t,u_t,\omega)$, as particular case $b=b(t,x_t,u_t,\xi_t)$, where $\xi_t$ is some other process. In Pham's book they consider only case where $b=b(t,x_t,u_t)$ and $b$ is deterministic, and that's not exactly what I'm looking for. I am a little bit familiar with maximum principle and FBSDEs, but never heard about Peng-Wu's result. I would appreciate if you gave me references on it. – niyazets Nov 5 '12 at 8:00
In this case the value function becomes an adapted process and the HJB equation a BSPDE, see Peng 1992. – user39563 Sep 4 '13 at 20:44

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.