2
$\begingroup$

Let $A$ and $B$ be any real matrices. I would like to find the solution of a linear system $Ax=B$ using the SVD decomposition of $A$ given by $A = U S V^t$. If I am not very wrong, I believe I can obtain this solution using

$\ \ \ X = VL^tU^tB$

in which

$\ \ \ L_{ij} = \begin{cases} 1/S_{ij} && \text{if} S_{ij} \not= 0 \\\\ 0 && \text{otherwise} \end{cases} $

This seems to work. However, for the case when $A$ has more columns than rows, I would like to find not only the best solution (in the least squares sense), but also the solution which has minimum norm. How can I do that?

$\\\\$

For example, by taking

$ \ \ \ A = \begin{pmatrix} 1 & 2 & 3 \\\\ 4 & 5 & 6 \end{pmatrix} \ \ \ \ \text{and} \ \ \ B = \begin{pmatrix} 7 & 8 & 9 \\\\ 10 & 11 & 12 \end{pmatrix} $

The aforementioned method gives the solution

$\ \ \ \hat{X} = \begin{pmatrix} -2.1667 & -2.6667 & -3.1667 \\\\ 0.3333 & 0.3333 & 0.3333 \\\\ 2.8333 & 3.3333 & 3.8333 \\\\ \end{pmatrix} $

I can verify this is a valid solution by taking

$\ \ \ A\hat{X} = \begin{pmatrix} 7 & 8 & 9 \\\\ 10 & 11 & 12 \end{pmatrix}$

However, I am not sure this is the minimum norm solution.

Improve this question
$\endgroup$
2
  • $\begingroup$ You're missing a transpose in $X = VL^tU^tB$. $\endgroup$
    – Federico Poloni
    Commented Jun 26, 2012 at 7:20
  • $\begingroup$ Thanks for pointing it out. By the way, I just realized I should have asked this on math.stackexchange instead. Didn't realized there was a different site for more trivial questions. $\endgroup$
    – César
    Commented Jun 27, 2012 at 16:51

1 Answer 1

Reset to default
4
$\begingroup$

Take a look at the wikipedia page on the Moore-Penrose pseudoinverse, specifically Sections 5.3, 6.1, and 6.3.

Improve this answer
$\endgroup$
2
  • $\begingroup$ So it seems the solution I mentioned is indeed the least squares solution then. Is that correct? $\endgroup$
    – César
    Commented Jun 26, 2012 at 2:09
  • 2
    $\begingroup$ Yes, it is. ---- $\endgroup$
    – Federico Poloni
    Commented Jun 26, 2012 at 7:20

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .