Optimization version of the Sylvester equation - MathOverflow

Optimization version of the Sylvester equation

2012-12-26T15:40:29Z

The Sylvester equation is a matrix equation of the form $AX-XB=C,$ where $A,B,C$ are given matrices of dimension $m\times m,n\times n$ and $m\times n$ and $X$ is an unknown matrix of dimension $m\times n.$ It is a well known fact that the equation has an unique solution if and only if the matrices $A$ and $B$ have disjoint spectrum. If they do not have disjoint spectrum, then the result in general depends on $C.$

While determining perturbation of eigenvalues in certain context I was naturally drawn to the the problem of determining the minimum, $min_{X}||AX-XB-C||,$ where $||.||$ is the Frobenius norm. Clearly, if the spectrum of $A$ and $B$ is disjoint then there is a choice of $X$ for which the norm is zero. Otherwise, we need to resort to certain optimization techniques. One approach could be to vectorize the matrices using Kronecker products and determine the minimum of a linear system.

The problem is: "What is the choice of $X$ for which the norm $||AX-XB-C||$ attains minimum (if it exists) when the spectra of $A$ and $B$ are not disjoint?"

I have not found any literature on discussion about similar problems. I would be very thankful for any references or suggestions in this direction.

Answer by Betrand for Optimization version of the Sylvester equation

2012-12-26T15:53:28Z

Instead of minimizing the norm, the following note proposes a conjecture on minimizing the rank of $AX-XB-C$.

M. Lin, H. Wimmer, The generalized Sylvester matrix equation, rank minimization and Roth's equivalence theorem , Bull. Aust. Math. Soc. 84 (2011) 441-443. http://www.math.uwaterloo.ca/~m29lin/LW2011.pdf

Answer by S. Sra for Optimization version of the Sylvester equation

2012-12-27T11:25:42Z

First recall two basic ideas.

Lemma. Let $A$, $B$, $C$ be arbitrary; then, $\text{vec}(ABC) = (C^T \otimes A)\text{vec}(B)$, where $\otimes$ denotes the Kronecker product and $\text{vec}(\cdot)$ denotes the 'vec' operator that stacks columns of a matrix to obtain a long vector.

Notation. For any matrix $Z$, I will denote by its lowercase version $z$ the vector $\text{vec}(Z)$.

Now observe that $\|X\|_F^2 = \text{trace}(X^TX) = x^Tx$. Thus, (also essentially noted by the OP) we may rewrite the optimization problem as

\begin{equation*} \min_x \|Hx-c\|^2, \end{equation*} where $H = I \otimes A - B^T \otimes I$.

This equation may or may not have a unique solution, but the unique least-norm vector $x$ that solves it is given by $x=H^+c$, where $H^+$ is the Moore-Penrose pseudoinverse of $H$. Thus, clearly, the optimization problem has a solution, which answers the question as asked.

Of course, due to the numerical expense of computing the above solution (via the truncated SVD of $H$) might be too high. The OP might be interested in consulting the numerical analysis literature on how to deal with such a case.