I've set up a system of stochastic differential equations that I'd like to control. I'm new to optimal control theory and SDEs (and, admittedly, weak on PDEs), so I'm not certain if I've set this up correctly. If someone could let me know if I've not completely bungled it and how to set it up correctly, I'd be very grateful.
OK, I have the system of SDEs $$ dX_t = a(X_t,Y_t,u(t)) + \sigma(X_t,Y_t)dW_t,$$ $$ dY_t = b(X_t,Y_t,u(t)),$$
where $X_t$ and $Y_t$ are stochastic variables, $u(t) \in [0,1]$ is the control, and $dW_t$ is a Weiner process. Note that $u$ is only in $a$ and $b$. My goal is to find $$\min_{u(t)} J[u(t)] = E \left[\int_0^\infty Y_tdt \right],$$ where $E[\cdot]$ is the expected value.
So, I define the Hamiltonian $$H(x,y,p,q,r,u) = a(x,y,u)p + b(x,y,u)q + \sigma(x,y)r - y,$$ and get the adjoint SDEs $$dP_t = \frac{\partial H(x,y,P_t,Q_t,R_t,u)}{\partial x}dt + R_tdW_t,$$ $$dQ_t = \frac{\partial H(x,y,P_t,Q_t,R_t,u)}{\partial y}dt.$$
From the state and adjoint systems, I find the forward and backward Kolmogorov equations, where $\phi(x,y,t)$ and $\psi(x,y,p,q,r,t)$ are evolving probability distributions. These are: $$\frac{\partial \phi}{\partial t} = \frac{\partial^2}{\partial x^2}\left\{ \frac{\sigma^2}{2}\phi\right\} - \frac{\partial}{\partial x}\{ a\phi \} - \frac{\partial}{\partial y} \{b \phi\},$$ $$\frac{\partial \psi}{\partial t} = \frac{\partial H}{\partial x}\frac{\partial \psi}{\partial p} + \frac{\partial H}{\partial y}\frac{\partial \psi}{\partial q} - \frac{r^2}{2}\frac{\partial^2 \psi}{\partial r^2}.$$
$x\in[0,1]$ and $y\in[0,1]$. My initial condition is where $x=0$ and $y=1$. I think that the initial condition for the adjoint equation is the steady state solution of $\psi$.
I should note that $H$ is linear with respect to $u$, and thus this is bang-bang control. Further, if it's easier, I could use a different cost functional. I could try to find the time, $T$, such that $E[Y_T] = 0$.
I've been reading through Yong's Stochastic Controls: Hamiltonian Systems and HJB Equations to set this up. The next step is to solve this numerically, but I want to be sure this makes sense.
Thank you so much.
