I have a dynamic programming problem with an underlying diffusion $$ d X_t = \mu \, dt + d b_t$$ where $b_t$ is a standard brownian motion.

The HJB equation for the value function $v(x,t)$ I get is the following: \begin{align*} \max_{a \in [0,1]} a [ 2(1-v(x,t)) - 1] + \mu v_x(x,t) - v_t(x,t) + \frac12 v_{xx}(x,t) = 0 \end{align*}

The initial condition I have is $v(x,0) = 1_{x\ge 0}$.

Naturally, the optimal policy would be: \begin{align*} a = \begin{cases} 1 & \text{ if } 2(1-v) -1 > 0 \Leftrightarrow v < \frac12 \\ 0 & \text{ otherwise } \end{cases} \end{align*}

My natural inclination is to split the HJB equation into two cases when $a=1$ and when $a=0$. It is easy to make some manipulations to turn each of those PDEs into the heat equation $$u_t = \frac12 u_{xx}$$ with appropriately modified initial conditions.

What I am clueless about, after this, is how do I solve these! From what I read, it seems to be a free boundary problem. But, are there any methods to analytically solve these two equations pretending to take the boundary as given and then joining the two solutions smoothly? From whatever optimal control I know, that is the approach I would follow if I had ODEs. Any help would be extremely appreciated. Thanks.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.