Skip to main content
2 of 2
deleted 329 characters in body
Mark
  • 169
  • 6

Optimal Control / Hamilton-Jacobi-Bellman Equation

I face the following optimal control problem: let $X=(X_{1},X_{2})^{\top}$ be a controlled (Ito-)process with dynamics

$dX_{1}(t)=\big(X_{1}(t)\,\mu_{1} + a(t)\big)\;dt + X_{1}(t)\,\sigma_{1}\;dW_{1}(t)$

$dX_{2}(t)=\big(X_{2}(t)\,b(t)\,\mu_{2} - a(t)\big)\;dt + X_{2}(t)\,b(t)\,\sigma_{2}\;dW_{2}(t)$

where $a$ and $b$ are the two controls, $W_{1}$ and $W_{2}$ are two Brownian motions (uncorrelated for the sake of simplicity) and $\mu_{1}$, $\mu_{2}$, $\sigma_{1}$ and $\sigma_{2}$ are some (positive) constants.

I would now like two find an optimal pair of controls which solves

$\Phi(t,x) \;=\; \max_{(a,b)}\;\mathbb{E}\bigg[\; U_{1}\big(X_{1}(T)\big) + U_{2}\big(X_{2}(T)\big) \;\bigg|\; \mathcal{F}_{t} \;\bigg]$

where $U_{1}$ and $U_{2}$ are two suitable utility funtions. Deriving the HJB equation for this problem I arrived at

$\sup_{(a,b)}\;\Big\{\; \partial_{t}\Phi + (\partial_{x_{1}}\Phi)\,(X_{1}\,\mu_{1}+a) + (\partial_{x_{2}}\Phi)\,(X_{2}\,\mu_{2}\,b-a) + \frac{1}{2}(\partial_{x_{1}x_{1}}\Phi)(X_{1}\,\sigma_{1})^{2} + \frac{1}{2}(\partial_{x_{2}x_{2}}\Phi)(X_{2}\,\sigma_{2}\,b)^{2} \;\Big\} = 0$

where I left out the arguments for notational convience. The corresponding boundary condition is

$\Phi(T,x) = U_{1}(x_{1}) + U_{2}(x_{2})$

What stuns me is, that when deriving the corresponding first-order conditions, the control $a$ "drops out":

$\partial_{x_{1}}\Phi - \partial_{x_{2}}\Phi = 0$

$(\partial_{x_{2}}\Phi)\,x_{2}\,\mu_{2} + (\partial_{x_{2}x_{2}}\Phi)\,(x_{2}\,\sigma_{2})^{2}\,b = 0$

At this point I don't know how to proceed, because I am not able to derive the corresponding optimal control $a$ in terms of the value process (or its derivatives).

Any comments, ideas and suggestions are highly appreciated. Thank you!

Mark
  • 169
  • 6