# Deriving HJB equation (why $\frac{dZ_t}{dt}=0$?)

I am trying to derive the HJB equation in a stochastic setting. Let me exemplify my problem with the simplest case where there is no control, just one state variable. Assume the payoff is given by $$W(X_{t})\equiv E_{t}\left\{ \int_{t}^{\infty}e^{-\rho(s-t)}u(X_{s})ds\right\}$$ where $X_{t}$ is given by $$dX_{t}=\mu(X_{t},t)dt+\sigma(X_{t},t)dZ_{t}$$ and $Z_{t}$ is the standard Brownian Motion. For any $dt>0$ we can write: $$W(X_{t})=E_{t}\left\{ \int_{t}^{t+dt}e^{-\rho(s-t)}u(X_{s})ds+e^{-\rho dt}W(X_{t+dt})\right\}$$

$$\left(1-e^{-\rho dt}\right)W(X_{t})=E_{t}\left\{ \int_{t}^{t+dt}e^{-\rho(s-t)}u(X_{s})ds+e^{-\rho dt}\left[W(X_{t+dt})-W(X_{t})\right]\right\} \tag{1}$$

From Ito calculus we get that (and assuming that $W(\cdot)$ is well behaved): $$W(X_{t+dt})-W(X_{t})=\int_{t}^{t+dt}W'(X_{s})dX_{s}+\frac{1}{2}\int_{t}^{t+dt}W''(X_{s})d[X_{s}]=\int_{t}^{t+dt}W'(X_{s})dX_{s}+\frac{1}{2}\int_{t}^{t+dt}\sigma(X_{t},t)W''(X_{s})ds$$ where the last equality follows from the known properties of the quadratic variation of the process $X_{t}$. Plugging this back in (1): $$\left(1-e^{-\rho dt}\right)W(X_{t})=E_{t}\left\{ \int_{t}^{t+dt}e^{-\rho(s-t)}u(X_{s})ds+e^{-\rho dt}\left[\int_{t}^{t+dt}W'(X_{s})dX_{s}+\frac{1}{2}\int_{t}^{t+dt}\sigma(X_{t},t)W''(X_{s})ds\right]\right\}$$

Dividing both sides by $dt$ and taking the limit $dt\rightarrow0$: $$\rho W(X_{t})=E_{t}\left\{ u(X_{t})+\lim_{dt\rightarrow0}\frac{\int_{t}^{t+dt}W'(X_{s})dX_{s}}{dt}+\frac{1}{2}\sigma(X_{t},t)W''(X_{t})\right\}$$ where I used the fact that when dealing with the Riemann integral: $\lim_{dt\rightarrow0}\frac{\int_{t}^{t+dt}f(x_{s})ds}{dt}=f(x_{t})$ (from standard calculus).

As you can see, I am almost there. I just don't know how to deal with term $\lim_{dt\rightarrow0}\frac{\int_{t}^{t+dt}W'(X_{s})dX_{s}}{dt}$. For example, assume $\mu(X_{t},t)=0$ and $\sigma(X_{t},t)=1$, so that $X_{t}$ is simply the standard Brownian Motion $Z_{t}$. In that case, to get the HJB formula right I would need: $$\lim_{dt\rightarrow0}\frac{\int_{t}^{t+dt}W'(Z_{s})dZ_{s}}{dt}=0$$ But I don't know how to prove that this is true. More generally (for any $\mu(X_{t},t)$ and $\sigma(X_{t},t)$), I would need to prove: $$\lim_{dt\rightarrow0}\frac{\int_{t}^{t+dt}W'(X_{s})dX_{s}}{dt}=\mu(X_{t},t)W'(X_{t})$$ which I am also not sure how to do. Any ideas?