Solve the optimal control problem of the LQR kind $$ \min_u \int_0^{+\infty} x_1^2+x_2^2+\gamma(u_1^2+u_2^2) \, dt \quad\text{such that}\quad \begin{cases}\dot x_1=\alpha(x_2-x_1)+u_1,& x_1(0)=1,\\\dot x_2=\beta(x_1-x_2)+u_2,& x_2(0)=-1\end{cases} $$ where $\alpha>\beta>0$ and $\gamma>0$.

I notice that all $x_i$ and $u_i$ in the integrand are squared and that there are no subtractions, hence the integrand has a minimum value in $0$ reached when $x_i=u_i=0$. Moreover, since $x_i$ and $u_i$ are squared, I guess the integrand is a paraboloid, hence a convex function. Could this information be helpful in finding a solution?

Since I'm not able to find a trivial solution via heuristic arguments, I tried using some methods (based on Riccati equation).

First method

Introducing $Q=B=I_2,\ \ R=\gamma I_2,\ \ A=\begin{pmatrix}-\alpha & \alpha\\\beta & -\beta\end{pmatrix}$, the problem can be rewritten as $$ \min_u \int_0^\infty (x^T Q x + u^TRu) \, dt \quad\text{such that}\quad \begin{cases}\dot x = Ax+Bu \\ x(0) = \begin{pmatrix}1\\-1\end{pmatrix}\end{cases} $$ with the following associated Riccati equation \begin{align*}\tag1 0 &= Q+A^TS+SA-SBR^{-1}BS,\qquad S=\begin{pmatrix}s_1 & s_2 \\ s_2 & s_3\end{pmatrix} \\ &= I+A^TS+SA-\frac1\gamma S^2 \end{align*} Both $Q$ and $R$ are symmetric and positive definite, so the optimal control is given by $$ u = -R^{-1}B^TSx = -\frac1\gamma Sx, $$ plugging it in the expression for $\dot x$ and integrating we can find $x$. But first we have to find $S$. Equation $(1)$ is equivalent to the following nonlinear system $$ \begin{cases} -\dfrac{s_1^2}{\gamma}-2\alpha s_1 -\dfrac{s_2^2}{\gamma}+2\beta s_2+1=0\\ \alpha s_1 - \alpha s_2 - \beta s_2 + \beta s_3 - \dfrac{s_1s_2}{\gamma}-\dfrac{s_2s_3}{\gamma}=0\\ -\dfrac{s_2^2}{\gamma}+2\alpha s_2 - \dfrac{s_3^2}{\gamma} - 2\beta s_3+1 = 0 \end{cases} $$ which apparently doesn't have a trivial solution. Since I don't know how to solve it by hand, I tried running the following Matlab code

syms x y z a b g
eqn1 = 0 == -x^2/g-2*a*x-y^2/g+2*b*y+1;
eqn2 = 0 == a*x-a*y-b*y+b*z-x*y/g-y*z/g;
eqn3 = 0 == -y^2/g+2*a*y-z^2/g-2*b*z+1;
sol = solve([eqn1, eqn2, eqn3], [x, y, z]);

but the provided solutions are huge in length, hence they are not handy.

Second method

Alternatively, $S$ could be found by determining the eigenvectors of the associated Hamiltonian matrix $$ H = \begin{pmatrix}A & -BR^{-1}B^T \\ -Q & -A^T \end{pmatrix} = \begin{pmatrix}A & -\gamma^{-1}I_2 \\ -I_2 & -A^T \end{pmatrix} $$ whose special feature is that if $λ$ is an eigenvalue, then also $−λ$, $\bar λ$ and $−\bar λ$ are eigenvalues. Moreover, denoting $\begin{pmatrix}V_1 \\ V_2\end{pmatrix}\in\mathbb C^{4\times2}$ the matrix whose two columns are the eigenvectors corresponding to the two eigenvalues of $H$ having negative real part (more info here), we have $$ S = V_2 V_1^{-1} $$ Apparently there is no explicit decomposition of $H$ (a decomposition is found here, however without explicit formulas for the eigenvalues and eigenvectors). The Matlab function icare finds $S$ using this method, but only if the parameters are known

syms a b g
B=eye(2); Q=eye(2); R=g*eye(2); A=[-a a;b -b];

Error using icare
Conversion to logical from sym is not possible.

Neither method seems to provide a handy expression for the solution. Do you how to solve the problem using the two methods above or other methods?

  • 1
    $\begingroup$ Why do you believe a handy expression exists? In other words, why so you ecpect a simpler expression than the result of your method 1. $\endgroup$ – Piyush Grover Apr 1 '20 at 23:03
  • $\begingroup$ @PiyushGrover I guess a handy expression exists since the integrand is convex. What expression are you talking about? I did not write any expression for $x$ $\endgroup$ – sound wave Apr 2 '20 at 1:51
  • 1
    $\begingroup$ What does the problem being convex have to do with "handy expression" ? Convex just means you get a global minima. The expression I was referring to is the output of matlab from method 1. $\endgroup$ – Piyush Grover Apr 2 '20 at 3:44
  • 2
    $\begingroup$ Please do not double-post. Solve a 2-dimensional optimal control problem via Riccati nonlinear equation $\endgroup$ – Federico Poloni Apr 2 '20 at 6:30
  • $\begingroup$ @FedericoPoloni should I close this one, and update the old one adding the text written here? $\endgroup$ – sound wave Apr 2 '20 at 8:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.