Generalizing HJB equation for a terminal stopping time

Question

The following is one version of the Hamilton–Jacobi–Bellman (HJB) equation:

Suppose we have a Brownian motion $W$ and a counting process $N$ with a stochastic intensity $\lambda$ on a time interval $[0,T]$. We have a given set of admissible controls $\mathcal A$ which take values in a set $U\subset\mathbb R$ and we control the intensity $\lambda$. Let $X$ be a process on $[0,T]$ with dynamics

$$dX^u_s = \mu ds + \sigma dW_s + \gamma dN^u_s$$ $$X_t=x,$$

where $x,\mu,\sigma,\gamma\in\mathbb R$, $t\in [0,T]$, $u\in \mathcal A$.

Consider an optimization problem of the form

$$ V(t,x) = \sup_{u\in \mathcal A} E\left( g(X^u_T) +\int_t^T f(X^u_s,u_s)ds\right).$$

Under certain assumptions the previous value function is a (viscosity) solution of the HJB equation:

$$v_t + \mu v_x + \frac{1}{2}\sigma^2v_{xx} + \sup_{u\in U} \left\{\lambda^u(t)(v(t,x+\gamma) - v(t,x)) + f(x,u)\right\}=0$$

$$v(T,.) = g(.) $$

My question is: how would the previous equation (and its terminal condition) change if we were to change the terminal time $T$ for a stopping time $\tau\leq T$ in the definition of the value function. i.e.,

$$ V(t,x) = \sup_{u\in \mathcal A_t} E\left( g(X^u_{\tau}) +\int_t^{\tau} f(X^u_s,u_s)ds\right).$$

Can anything be said in this situation?

What if we added the hypothesis of $\tau$ being absorbent for $X$? i.e., $X$ is constant over $[\tau,T]$.

Any reference would be very much appreciated. Oksendal-Sulem's book entitled "Applied Stochastic Control of Jump Diffusions", for example, as comprehensive as it is, does not deal with value functions with time dependence.

Answer 1

At least formally, the given HJB equation with possibly some additional Dirichlet boundary conditions (more on this point below) does hold for the value function associated to the stopped or absorbed process. Recall that the main tool in the (formal) derivation of HJB is Ito's change of variables formula for semimartingales; see, e.g., Theorem 32 in Chapter II of P. Protter's book Stochastic Integration and Differential Equations. (Note that the value function is required to be $C^2$ in space and $C^1$ in time in order to apply Ito's formula.) Unfortunately, the resulting equations (including the initial/boundary conditions) do not generally have a classical sense solution, and as the OP points out, one often needs to weaken the notion of solution.

Special case of a Controlled Process that is Absorbed upon Leaving a Set.

For any $u \in \mathcal{A}$ and $t>0$, the (time inhomogeneous) infinitesimal generator for the solutions of the controlled SDE is simply: $$ L^u_t q(x) = \underbrace{\mu \partial_x q(x) + \frac{1}{2} \sigma^2 \partial_{xx} q(x)}_{\text{continuous part}} + \overbrace{\lambda^u_t (q(x+\gamma) - q(x))}^{\text{discontinuous part}} $$ The fact that the controlled solutions possess this infinitesimal generator is a key ingredient to (formally) deriving the HJB equation. Suppose further that the process is stopped when it first exits say an interval $D \subset \mathbb{R}$. Then a formal application of the previously mentioned Ito's formula to the value function yields: $$ \partial_t v + \sup_{u \in \mathcal{A}} \left\{ L_t^u v_t + f(x,u)\right\} = 0 \qquad \text{for $x \in D$ and $t \in [0, T)$} $$ with the additional "initial" and boundary conditions: \begin{align*} &v(T,x) = 0 & \text{for $x \in D$} \\ &v(t,x) =g(x) & \text{for $x \in \partial D$ and $ t\in [0, T)$} \end{align*} In this case, the stopping time is explicitly $\tau = T \wedge \inf\{ s \in [t,T]: X_s^u \notin D,~ X_t^u = x \}$. Note that one can pull the continuous part of the generator out of the sup in the above HJB equation, since it does not depend on $u$, but I left it in there for the sake of readability.

Reference

For related but slightly different work, see Controlled Variance and Jumps (Chapter 13) of:

Harold J. Kushner and Paul Dupuis. Numerical Methods for Stochastic Control Problems in Continuous Time. Springer-Verlag, Second Edition 2001, First Edition 1992.

It's slightly different from the OP's problem because the control they consider enters the jumps themselves rather than the jump intensities.