I have two questions on numerical methods for solving Hamilton-Jacobi-Bellman (HJB) equations with state constraints.

  1. Consider an optimal control problem given by $$ v(x) = \max_{\{u(t)\}_t} \int_o^\infty e^{-\rho t}r(x(t), u(t))\, dt, \quad x(0) = x, $$ subject to the law of motion $$\dot x = f(x(t),u(t))$$ and the state constraint $$g(x) \ge 0. \tag1$$ The HJB equation associated with this problem is given by $$ \rho v(x)= \max_{u} r(x,u) + \nabla v(x) \cdot f(x,u),\tag2 $$ where the state constraint (1) is imposed by requiring that $v$ is subsolution of (2) at the boundary points $\bar x \in \{x : g(x)=0\}$.

The types of problems I am working on are typically solved numerically by the so called "upwind scheme" using finite difference methods. My question is, how to incorporate the state constraint $g(x) \ge 0$ in these (or some other) numerical schemes. I have seen some papers considering one state and one control variable, but the methods in these papers seem to be quite 1-dimensional; I would need to handle a slightly more general case with two states and two controls. Thus, I am looking for a clear and comprehensive reference for the numerical (finite difference) methods handling this problem.

  1. To make the problem slightly more interesting, consider a sort of free boundary problem, where (1) is replaced by a more general constraint $g(x,u)\ge 0$ depending also on the control variable $u$. Is there any theory for such problems (existence, uniqueness), and how about numerical methods for solving the problem? What would be the reference?

This question arises from macroeconomic models of the housing market, when imposing loan-to-value (LTV) collateral constraint of type $B \le \theta PH$, where $B$ denotes the household debt, $H$ denotes the amount of housing, $\theta$ is a parameter describing maximal LTV ratio and $P=P(H, u)$ is the (endogenously determined) price of housing, given the state $x=(H,B)$; here $$g(x,u) = \theta PH-B.$$

  • $\begingroup$ You may want to ask this on scicomp.SE. $\endgroup$ – David Ketcheson Jan 16 at 22:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.