Solve equation with matrix variable - MathOverflow most recent 30 from http://mathoverflow.net 2013-05-23T19:14:45Z http://mathoverflow.net/feeds/question/114605 http://www.creativecommons.org/licenses/by-nc/2.5/rdf http://mathoverflow.net/questions/114605/solve-equation-with-matrix-variable Solve equation with matrix variable Wei Liu 2012-11-26T23:21:07Z 2012-11-27T07:13:59Z <p>I want to solve a matrix $\Omega$ from a equation $\sum_k (\Omega + \Theta_k)^{-1} = Q$. The $Q$ and $\Theta, \forall k=1...K$ are known, and are positive definite matrices. $\Omega$ also has to be positive definite. all matrices are large (a few thousands of columns and rows). My questions are:</p> <p>(1) Is there a closed-form solution? How do I simplify the sum of the inverse of two matrix sum?</p> <p>(2) I'm OK to go for a numerical solution. But how do I define this problem? An optimization problem to minimize something like $f(\Omega) = ||\sum_k (\Omega + \Theta_k)^{-1} - Q||$? Do I need to minimize the frobenius norm, (just like minimizing the L-2 norm in a least square problem)? Considering the constraint that $\Omega$ is positive definite, can I solve it by semi-definite programming? How do I redefine the problem in a linear/semi-definite programming? I don't have much knowledge of linear programming. I would prefer a general gradient descent rather than LP. But I'm OK to use LP if I know how to do.</p> <p>This problem comes from the estimation of inverse covariance matrix of multi-variate Gaussian distribution. </p> <p>EDIT: Both $\Theta_k$ and $\Omega$ are sparse, if that helps.</p> http://mathoverflow.net/questions/114605/solve-equation-with-matrix-variable/114636#114636 Answer by S. Sra for Solve equation with matrix variable S. Sra 2012-11-27T07:13:59Z 2012-11-27T07:13:59Z <p>Here is a partial solution to the first question in the original post. Let's look at the equation $$\tag{1} \sum\nolimits_{i=1}^m (X+ \Theta_i)^{-1} = Q.$$</p> <p><strong>Lemma (Existence).</strong> If all $\Theta_i$ are (strictly) positive definite, then (1) has a positive semidefinite solution only if $Q \preceq \sum_i \Theta_i^{-1}$.</p> <p><em>Proof.</em> Suppose $Q=\sum_i \Theta_i^{-1}$, then clearly $X=0$ is the solution. Since, $(X+\Theta_i)^{-1} \preceq \Theta_i^{-1}$ for any $X \succeq 0$, on summing up we see that $Q \preceq \sum_i \Theta_i^{-1}$ must hold. Moreover, in this case if there is a solution, then it must be strictly positive definite. A little extra argument shows that in this case, there must exist a unique positive definite solution.</p> <p>This lemma shows that in case $Q$ does not satisfy the requirement, the original equation has no solution, and it might be preferable to minimize $\|\sum_i (X+\Theta_i)^{-1}-Q\|_F^2$ instead.</p> <p><strong>Lemma (Bounds).</strong> Any feasible solution to (1) must lie in the set $\Omega := [0, mQ^{-1}]$.</p> <p><em>Proof.</em> The lower bound $X \succeq 0$ is obvious. Following an argument similar to the previous lemma, we see that $Q=\sum_i (X+\Theta_i)^{-1} \preceq \sum_i X^{-1}$, which implies that $m X^{-1} \succeq Q$, or equivalently, $X \preceq m Q^{-1}$.</p> <p><strong>Idea</strong> Now that we have a compact set $\Omega$, we just need to setup a strictly contractive nonlinear map $G : \Omega \to \Omega$. I have not proved strict contraction of the map below, but numerically it seems to work. As one might suspect from the above lemmas, the rate of convergence depends on $\|Q-\sum_i \Theta_i^{-1}\|$, so that for small values of this quantity, the iteration converges more slowly.</p> <p>Suppose, that $X \succ 0$. Denote by $S^{++}$ the set of $n\times n$ strictly positive definite matrices. Then, define the nonlinear map $\mathcal{G} : S^{++} \to S^{++}$ as \begin{equation*} \mathcal{G} = X \mapsto X^{1/2}\left(\sum\nolimits_{i=1}^m Q^{-1/2}(X+\Theta_i)^{-1}Q^{-1/2}\right)X^{1/2}. \end{equation*}</p> <p><strong>TODO</strong> If I get time, I might think about proving that the above map generates convergent solutions. Or one can come up with some other fixed point iteration.</p>