2
$\begingroup$

I am trying to solve the following Bellman Equation:

$V(s) =\max_u \left[a'u - (u-s)'Q(u-s) + V(u)\right]$

In the equation above, $s,u,a\in \mathbb R^n$, $Q\in \mathbb R^{n\times n}$ is positive semidefinite. The equation above is very similar to the solution equation for an infinite-horizon, discrete-time Linear-Quadratic Regulator Problem. It can be restated as

$V(s) =\max_u \left[a'(u+s) - u'Qu + V(s+u)\right]$

However, there are linear terms in $s$ and $x$. My question is whether anyone knows of a reference in which the LQR problem is studied in the case of linear terms in the cost function.

Improve this question
$\endgroup$
3
  • $\begingroup$ In the LQR problems I have studied $s$ and $u$ are either sequences or functions of a real variable. Is this the case here, too? Could you please write the full equations, if so? $\endgroup$
    – Federico Poloni
    Commented Apr 21, 2015 at 6:14
  • $\begingroup$ The question as asked is unclear. There is no $x$ in your equations. $u$ is typically the control input and $x$ is typically the state. Are you using $V$ to mean the cost-to-go/value function? What does the expression $u+s$ mean? $\endgroup$
    – Gus
    Commented Apr 24, 2015 at 20:16
  • $\begingroup$ The formulation is unambiguous. It's not true that $x$ Is always used as a state variable (in OR literature $s$ is common, in EE control literature $x$ is common), and the choice of a simbol is irrelevant. $u$ and $s$ live in the same space, and sum is elementwise on this space. Yes, $V$ is a cost-to-go. Which has nothing to do with clarity of formulation. $\endgroup$
    – gappy3000
    Commented Apr 25, 2015 at 21:36

0

Reset to default

You must log in to answer this question.

Browse other questions tagged .