# Solve optimal control problem whose associated system is nonlinear

Solve the optimal control problem of the LQR kind $$\min_u \int_0^{+\infty} x_1^2+x_2^2+\gamma(u_1^2+u_2^2) \, dt \quad\text{such that}\quad \begin{cases}\dot x_1=\alpha(x_2-x_1)+u_1,& x_1(0)=1,\\\dot x_2=\beta(x_1-x_2)+u_2,& x_2(0)=-1\end{cases}$$ where $$\alpha>\beta>0$$ and $$\gamma>0$$.

I notice that all $$x_i$$ and $$u_i$$ in the integrand are squared and that there are no subtractions, hence the integrand has a minimum value in $$0$$ reached when $$x_i=u_i=0$$. Moreover, since $$x_i$$ and $$u_i$$ are squared, I guess the integrand is a paraboloid, hence a convex function. Could this information be helpful in finding a solution?

Since I'm not able to find a trivial solution via heuristic arguments, I tried using some methods (based on Riccati equation).

## First method

Introducing $$Q=B=I_2,\ \ R=\gamma I_2,\ \ A=\begin{pmatrix}-\alpha & \alpha\\\beta & -\beta\end{pmatrix}$$, the problem can be rewritten as $$\min_u \int_0^\infty (x^T Q x + u^TRu) \, dt \quad\text{such that}\quad \begin{cases}\dot x = Ax+Bu \\ x(0) = \begin{pmatrix}1\\-1\end{pmatrix}\end{cases}$$ with the following associated Riccati equation \begin{align*}\tag1 0 &= Q+A^TS+SA-SBR^{-1}BS,\qquad S=\begin{pmatrix}s_1 & s_2 \\ s_2 & s_3\end{pmatrix} \\ &= I+A^TS+SA-\frac1\gamma S^2 \end{align*} Both $$Q$$ and $$R$$ are symmetric and positive definite, so the optimal control is given by $$u = -R^{-1}B^TSx = -\frac1\gamma Sx,$$ plugging it in the expression for $$\dot x$$ and integrating we can find $$x$$. But first we have to find $$S$$. Equation $$(1)$$ is equivalent to the following nonlinear system $$\begin{cases} -\dfrac{s_1^2}{\gamma}-2\alpha s_1 -\dfrac{s_2^2}{\gamma}+2\beta s_2+1=0\\ \alpha s_1 - \alpha s_2 - \beta s_2 + \beta s_3 - \dfrac{s_1s_2}{\gamma}-\dfrac{s_2s_3}{\gamma}=0\\ -\dfrac{s_2^2}{\gamma}+2\alpha s_2 - \dfrac{s_3^2}{\gamma} - 2\beta s_3+1 = 0 \end{cases}$$ which apparently doesn't have a trivial solution. Since I don't know how to solve it by hand, I tried running the following Matlab code

syms x y z a b g
eqn1 = 0 == -x^2/g-2*a*x-y^2/g+2*b*y+1;
eqn2 = 0 == a*x-a*y-b*y+b*z-x*y/g-y*z/g;
eqn3 = 0 == -y^2/g+2*a*y-z^2/g-2*b*z+1;
sol = solve([eqn1, eqn2, eqn3], [x, y, z]);


but the provided solutions are huge in length, hence they are not handy.

## Second method

Alternatively, $$S$$ could be found by determining the eigenvectors of the associated Hamiltonian matrix $$H = \begin{pmatrix}A & -BR^{-1}B^T \\ -Q & -A^T \end{pmatrix} = \begin{pmatrix}A & -\gamma^{-1}I_2 \\ -I_2 & -A^T \end{pmatrix}$$ whose special feature is that if $$λ$$ is an eigenvalue, then also $$−λ$$, $$\bar λ$$ and $$−\bar λ$$ are eigenvalues. Moreover, denoting $$\begin{pmatrix}V_1 \\ V_2\end{pmatrix}\in\mathbb C^{4\times2}$$ the matrix whose two columns are the eigenvectors corresponding to the two eigenvalues of $$H$$ having negative real part (more info here), we have $$S = V_2 V_1^{-1}$$ Apparently there is no explicit decomposition of $$H$$ (a decomposition is found here, however without explicit formulas for the eigenvalues and eigenvectors). The Matlab function icare finds $$S$$ using this method, but only if the parameters are known

syms a b g
B=eye(2); Q=eye(2); R=g*eye(2); A=[-a a;b -b];
icare(A,B,Q,R)

Error using icare
Conversion to logical from sym is not possible.


Neither method seems to provide a handy expression for the solution. Do you how to solve the problem using the two methods above or other methods?

• Why do you believe a handy expression exists? In other words, why so you ecpect a simpler expression than the result of your method 1. Commented Apr 1, 2020 at 23:03
• @PiyushGrover I guess a handy expression exists since the integrand is convex. What expression are you talking about? I did not write any expression for $x$ Commented Apr 2, 2020 at 1:51
• What does the problem being convex have to do with "handy expression" ? Convex just means you get a global minima. The expression I was referring to is the output of matlab from method 1. Commented Apr 2, 2020 at 3:44
• Please do not double-post. Solve a 2-dimensional optimal control problem via Riccati nonlinear equation Commented Apr 2, 2020 at 6:30
• @FedericoPoloni should I close this one, and update the old one adding the text written here? Commented Apr 2, 2020 at 8:48