# Optimal Control / Hamilton-Jacobi-Bellman Equation

I face the following optimal control problem: let $X=(X_{1},X_{2})^{\top}$ be a controlled (Ito-)process with dynamics

$dX_{1}(t)=\big(X_{1}(t)\,\mu_{1} + a(t)\big)\;dt + X_{1}(t)\,\sigma_{1}\;dW_{1}(t)$

$dX_{2}(t)=\big(X_{2}(t)\,b(t)\,\mu_{2} - a(t)\big)\;dt + X_{2}(t)\,b(t)\,\sigma_{2}\;dW_{2}(t)$

where $a$ and $b$ are the two controls, $W_{1}$ and $W_{2}$ are two Brownian motions (uncorrelated for the sake of simplicity) and $\mu_{1}$, $\mu_{2}$, $\sigma_{1}$ and $\sigma_{2}$ are some (positive) constants.

I would now like two find an optimal pair of controls which solves

$\Phi(t,x) \;=\; \max_{(a,b)}\;\mathbb{E}\bigg[\; U_{1}\big(X_{1}(T)\big) + U_{2}\big(X_{2}(T)\big) \;\bigg|\; \mathcal{F}_{t} \;\bigg]$

where $U_{1}$ and $U_{2}$ are two suitable utility funtions. Deriving the HJB equation for this problem I arrived at

$\sup_{(a,b)}\;\Big\{\; \partial_{t}\Phi + (\partial_{x_{1}}\Phi)\,(X_{1}\,\mu_{1}+a) + (\partial_{x_{2}}\Phi)\,(X_{2}\,\mu_{2}\,b-a) + \frac{1}{2}(\partial_{x_{1}x_{1}}\Phi)(X_{1}\,\sigma_{1})^{2} + \frac{1}{2}(\partial_{x_{2}x_{2}}\Phi)(X_{2}\,\sigma_{2}\,b)^{2} \;\Big\} = 0$

where I left out the arguments for notational convience. The corresponding boundary condition is

$\Phi(T,x) = U_{1}(x_{1}) + U_{2}(x_{2})$

What stuns me is, that when deriving the corresponding first-order conditions, the control $a$ "drops out":

$\partial_{x_{1}}\Phi - \partial_{x_{2}}\Phi = 0$

$(\partial_{x_{2}}\Phi)\,x_{2}\,\mu_{2} + (\partial_{x_{2}x_{2}}\Phi)\,(x_{2}\,\sigma_{2})^{2}\,b = 0$

At this point I don't know how to proceed, because I am not able to derive the corresponding optimal control $a$ in terms of the value process (or its derivatives).

Any comments, ideas and suggestions are highly appreciated. Thank you!

• I guess an optimal control $a$ might be to "reallocate" between processes $X_{1}$ and $X_{2}$, such that the resulting change in terminal utility is maximized, i.e. $a(t)=U_{1}^{'}\big(X_{1}(t)\big) - U_{2}^{'}\big(X_{2}(t)\big)$. – Mark Jun 4 '16 at 13:34
• Indeed $a$ is just doing some reallocation, but for that only $\int_0^T a(s) \, ds$ matters. However this might not be chosen arbitrarily, but might depend on the constraints you have. To understand the problem better, could you provide some economic motivation? In particular, are $X_1$ and $X_2$ to be assumed always nonnegative? And more general it would be interesting to understand why you have two different utility functions for the different processes which are nevertheless additive. – Stephan Sturm Jun 6 '16 at 2:52