1
$\begingroup$

I'm currently reading this paper describing a numerical scheme for the approximating optimal policy of a stochastic control problem. However, I run into a confusion directly on the first page where there seems to be a "common abuse of notation".

The author write the stochastic control problem determined by: $$ \begin{aligned} v(t,x)= & sup_{a \in \mathcal{A}} \, \mathbb{E}^{t,x}\left[ \int_t^T f(X_s^a,a_s)ds + g(X_T^a) \right]\qquad 0\leq t< T \\ dX_t^a =& b(X_s^a,a_s)dt + \sigma(X_s^a,a_s)dW_s, \end{aligned} $$ where $\mathscr{A}$ a non-empty set of $A\subseteq \mathbb{R}^q$-valued controls.

They state that the value function for this problem satisfies the HJB equation written: $$ \begin{aligned} \partial_t v + \sup_{a \in A}\left[ b(x,a).D_xv + \frac1{2}\operatorname{tr}(\sigma\sigma^T(x,a)D_x^2v) + f(x,a,v,\sigma^T(x,a).D_xv) \right] = & 0 \qquad (t,x)\in [0,T)\times \mathbb{R}^d\\ v(T,x)= & g(x) \qquad x \in \mathbb{R}^d. \end{aligned} $$

Problem: My problem is that $f$ is used to denote two different functions? So how is f defined in the HJB equation (given that I know how to define it in the control problem) then?

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ Agree it is horrible notation. f is still the same f, but now implicitly depends on $\sigma$, $v$,and its gradient because the $a$ is taken to be $a(x,v,\nabla v,\Delta v,\sigma)$. See these notes e.g.: columbia.edu/~dl3133/MFGSpring2018.pdf pages 63-70. In other words, $f=f(x,a(x,v,\nabla v,\Delta v,\sigma))$ $\endgroup$
    – Piyush Grover
    Commented Dec 13, 2019 at 16:52
  • $\begingroup$ In other words if my control doesn't depend on the value function explicitly then those terms may be ommited? Ex if it is constant? $\endgroup$
    – ABIM
    Commented Dec 13, 2019 at 20:40
  • 1
    $\begingroup$ Well, the optimality condition gives you HJB, where the Hamiltonian term (see notes I linked to) is maximized over all possible controls, i.e., to do the sup, you have to allow control to have arbitary dependence on $v,x,\sigma$.. So you cannot prescribe the optimal control to be constant: it comes out of the maximization (sup ) process. Now depending upon the system, it may depend just on $\nabla v$, .e.g., in linear quadratic case (example given in the paper you mention), and yes, then you can omit other terms $\endgroup$
    – Piyush Grover
    Commented Dec 13, 2019 at 20:56

0

Reset to default

You must log in to answer this question.