Take the 2-minute tour ×
MathOverflow is a question and answer site for professional mathematicians. It's 100% free, no registration required.

Let $A$ and $B$ be any real matrices. I would like to find the solution of a linear system $Ax=B$ using the SVD decomposition of $A$ given by $A = U S V^t$. If I am not very wrong, I believe I can obtain this solution using

$\ \ \ X = VL^tU^tB$

in which

$\ \ \ L_{ij} = \begin{cases} 1/S_{ij} && \text{if} S_{ij} \not= 0 \\\\ 0 && \text{otherwise} \end{cases} $

This seems to work. However, for the case when $A$ has more columns than rows, I would like to find not only the best solution (in the least squares sense), but also the solution which has minimum norm. How can I do that?

$\\\\$

For example, by taking

$ \ \ \ A = \begin{pmatrix} 1 & 2 & 3 \\\\ 4 & 5 & 6 \end{pmatrix} \ \ \ \ \text{and} \ \ \ B = \begin{pmatrix} 7 & 8 & 9 \\\\ 10 & 11 & 12 \end{pmatrix} $

The aforementioned method gives the solution

$\ \ \ \hat{X} = \begin{pmatrix} -2.1667 & -2.6667 & -3.1667 \\\\ 0.3333 & 0.3333 & 0.3333 \\\\ 2.8333 & 3.3333 & 3.8333 \\\\ \end{pmatrix} $

I can verify this is a valid solution by taking

$\ \ \ A\hat{X} = \begin{pmatrix} 7 & 8 & 9 \\\\ 10 & 11 & 12 \end{pmatrix}$

However, I am not sure this is the minimum norm solution.

share|improve this question
    
You're missing a transpose in $X = VL^tU^tB$. –  Federico Poloni Jun 26 '12 at 7:20
    
Thanks for pointing it out. By the way, I just realized I should have asked this on math.stackexchange instead. Didn't realized there was a different site for more trivial questions. –  César Jun 27 '12 at 16:51

1 Answer 1

up vote 3 down vote accepted

Take a look at the wikipedia page on the Moore-Penrose pseudoinverse, specifically Sections 5.3, 6.1, and 6.3.

share|improve this answer
    
So it seems the solution I mentioned is indeed the least squares solution then. Is that correct? –  César Jun 26 '12 at 2:09
1  
Yes, it is. ---- –  Federico Poloni Jun 26 '12 at 7:20

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.