Solving for Moore Penrose pseudo inverse - MathOverflow most recent 30 from http://mathoverflow.net 2013-06-19T04:51:35Z http://mathoverflow.net/feeds/question/72616 http://www.creativecommons.org/licenses/by-nc/2.5/rdf http://mathoverflow.net/questions/72616/solving-for-moore-penrose-pseudo-inverse Solving for Moore Penrose pseudo inverse WhitAngl 2011-08-10T19:33:14Z 2011-08-10T23:15:26Z <p>I have a system to solve, set up as : $$Ax = b$$ with a square rank deficient matrix $A$. The paper suggests to use a Moore Penrose pseudo inverse, which in my case can be computed using the traditional inverse : $$A^+ = (A+\frac{ee^T}{n})^{-1} - \frac{ee^T}{n}$$ where $e$ is a vector containing only ones, and $n$ is the dimension of the matrix. This matrix comes from the solution of a Multidimensional Scaling problem using the SMACOF method (the Guttman transform).</p> <p>However, in my case, my matrix $A$ is very sparse (and rank deficient) : what method can I use to efficiently solve the original system without making my matrix dense (as would be the case by using an SVD, by using the above formula for the pseudo inverse, by computing $A^TA$ or by QR factorization) ? $A$ is also symmetric, has a positive diagonal, and the other values are either -1 or 0, and such as the sum of each row (resp. column) is 0.</p> <p>Preferably, since I'll need to solve for multiple right hand sides with this same matrix, I would like to avoid performing the resolution from scratch for each right hand side. I would also like to get exactly the same result as the one obtained with the Moore Penrose pseudoinverse.</p> <p>Thanks.</p> http://mathoverflow.net/questions/72616/solving-for-moore-penrose-pseudo-inverse/72621#72621 Answer by Federico Poloni for Solving for Moore Penrose pseudo inverse Federico Poloni 2011-08-10T20:35:45Z 2011-08-10T20:35:45Z <p>For black-box linear algebra (GMRES and the like) you don't need "sparse", you only need "can compute products quickly". If you check the docs for your sparse solver, I'm sure there'll be a version where you can provide directly the function $v\mapsto Av$ (and sometimes $w\mapsto A^Tw$ is needed, too). In your case, you can compute $(A+\frac{ee^T}{n})v$ quickly by exploting sparsity.</p> <p>And yes, if you wish to use iterative solvers you'll have to solve the system more or less from scratch for every new right-hand side linearly independent from the previous ones. Some savings are possible, but getting them is still a hot research topic.</p> http://mathoverflow.net/questions/72616/solving-for-moore-penrose-pseudo-inverse/72627#72627 Answer by Igor Rivin for Solving for Moore Penrose pseudo inverse Igor Rivin 2011-08-10T21:54:50Z 2011-08-10T21:54:50Z <p>Are you looking for a fast practical method, or a fast theoretical method? If the former, there are very fast solvers based on sparse Cholesky or sparse LDL decomposition (both of which can be reused for many $b$). You should check out Tim Davis' beautiful book called something like "Sparse direct solvers".</p> <p>There are very few theoretical bounds, since the running time depends heavily on the sparsity pattern.</p>