I am trying to derive the HJB equation in a stochastic setting. Let me exemplify my problem with the simplest case where there is no control, just one state variable. Assume the payoff is given by $$ W(X_{t})\equiv E_{t}\left\{ \int_{t}^{\infty}e^{-\rho(s-t)}u(X_{s})ds\right\} $$ where $X_{t}$ is given by $$ dX_{t}=\mu(X_{t},t)dt+\sigma(X_{t},t)dZ_{t} $$ and $Z_{t}$ is the standard Brownian Motion. For any $dt>0$ we can write: $$ W(X_{t})=E_{t}\left\{ \int_{t}^{t+dt}e^{-\rho(s-t)}u(X_{s})ds+e^{-\rho dt}W(X_{t+dt})\right\} $$

$$ \left(1-e^{-\rho dt}\right)W(X_{t})=E_{t}\left\{ \int_{t}^{t+dt}e^{-\rho(s-t)}u(X_{s})ds+e^{-\rho dt}\left[W(X_{t+dt})-W(X_{t})\right]\right\} \tag{1} $$

From Ito calculus we get that (and assuming that $W(\cdot)$ is well behaved): $$ W(X_{t+dt})-W(X_{t})=\int_{t}^{t+dt}W'(X_{s})dX_{s}+\frac{1}{2}\int_{t}^{t+dt}W''(X_{s})d[X_{s}]=\int_{t}^{t+dt}W'(X_{s})dX_{s}+\frac{1}{2}\int_{t}^{t+dt}\sigma(X_{t},t)W''(X_{s})ds $$ where the last equality follows from the known properties of the quadratic variation of the process $X_{t}$. Plugging this back in (1): $$ \left(1-e^{-\rho dt}\right)W(X_{t})=E_{t}\left\{ \int_{t}^{t+dt}e^{-\rho(s-t)}u(X_{s})ds+e^{-\rho dt}\left[\int_{t}^{t+dt}W'(X_{s})dX_{s}+\frac{1}{2}\int_{t}^{t+dt}\sigma(X_{t},t)W''(X_{s})ds\right]\right\} $$

Dividing both sides by $dt$ and taking the limit $dt\rightarrow0$: $$ \rho W(X_{t})=E_{t}\left\{ u(X_{t})+\lim_{dt\rightarrow0}\frac{\int_{t}^{t+dt}W'(X_{s})dX_{s}}{dt}+\frac{1}{2}\sigma(X_{t},t)W''(X_{t})\right\} $$ where I used the fact that when dealing with the Riemann integral: $\lim_{dt\rightarrow0}\frac{\int_{t}^{t+dt}f(x_{s})ds}{dt}=f(x_{t})$ (from standard calculus).

As you can see, I am almost there. I just don't know how to deal with term $\lim_{dt\rightarrow0}\frac{\int_{t}^{t+dt}W'(X_{s})dX_{s}}{dt}$. For example, assume $\mu(X_{t},t)=0$ and $\sigma(X_{t},t)=1$, so that $X_{t}$ is simply the standard Brownian Motion $Z_{t}$. In that case, to get the HJB formula right I would need: $$ \lim_{dt\rightarrow0}\frac{\int_{t}^{t+dt}W'(Z_{s})dZ_{s}}{dt}=0 $$ But I don't know how to prove that this is true. More generally (for any $\mu(X_{t},t)$ and $\sigma(X_{t},t)$), I would need to prove: $$ \lim_{dt\rightarrow0}\frac{\int_{t}^{t+dt}W'(X_{s})dX_{s}}{dt}=\mu(X_{t},t)W'(X_{t}) $$ which I am also not sure how to do. Any ideas?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.