Is this formulation of the Singular Value Decomposition standard? - MathOverflow most recent 30 from http://mathoverflow.net 2013-05-19T17:24:06Z http://mathoverflow.net/feeds/question/43408 http://www.creativecommons.org/licenses/by-nc/2.5/rdf http://mathoverflow.net/questions/43408/is-this-formulation-of-the-singular-value-decomposition-standard Is this formulation of the Singular Value Decomposition standard? Dick Palais 2010-10-24T18:11:51Z 2011-01-07T13:43:46Z <p>In customary formulations of the Singular Value Decomposition or SVD that I have seen, (e.g., Wikipedia or Gil Strang's textbooks) it is always stated in terms of writing an $m \times n$ matrix $M$ (say of rank $r$) as a product $U \Lambda V$, where $U$ and $V$ are orthogonal $m \times m$ and $n \times n$ matrices and $\Lambda$ is a diagonal $m \times n$ matrix with non-negative "singular values" on the diagonal, $s_1 \ge s_2 \ge \ldots \ge s_r >0$ and the rest zero. Looking back over the many times I have taught linear algebra to both undergraduates and graduate students, I realized that I have not once covered the SVD, and even though I consider myself pretty knowledgeable about linear algebra, I have never felt comfortable with the statement of SVD or felt that I understood it in a more than formal way. (I might add that many theoretical linear algebra texts do not mention the SVD and many of my more theoretically minded colleagues do not even recognize the term.) But a few days ago, a social scientist friend of mine asked me about<br> a problem he was interested in; one that involved the SVD in an essential way. After thinking about it for a while, I realized that SVD can be reformulated as a statement about linear transformations that, to me at least, seems a lot more conceptual and geometric:</p> <p>If $T$ is a linear map, say of rank $r$, between finite dimensional inner-product spaces $V$ and $W$, then there are orthonormal bases $v_1, \ldots, v_m$ for $V$ and $w_1, \ldots, w_n$ for $W$ and $r$ positive numbers $s_1 \ge s_2 \ge \ldots \ge s_r$, such that $T v_i$ equals $s_i w_i$ if $i \le r$ and equals zero if $i > r$. </p> <p>I certainly realize that this is a pretty obvious reformulation of SVD, once you see it (and<br> those poor misguided souls who prefer matrices to linear transformations may even see it as a step backwards :-), but my question is whether there is some standard source for this reformulation that I can reference. </p> http://mathoverflow.net/questions/43408/is-this-formulation-of-the-singular-value-decomposition-standard/43410#43410 Answer by Mark Meckes for Is this formulation of the Singular Value Decomposition standard? Mark Meckes 2010-10-24T18:24:27Z 2010-10-24T18:24:27Z <p>This formulation, or something very close to it (I don't have the book with me) is in Axler's <em>Linear Algebra Done Right</em>.</p> http://mathoverflow.net/questions/43408/is-this-formulation-of-the-singular-value-decomposition-standard/43416#43416 Answer by Angelo for Is this formulation of the Singular Value Decomposition standard? Angelo 2010-10-24T19:14:18Z 2010-10-24T19:14:18Z <p>I just looked in Wikipedia (<a href="http://en.wikipedia.org/wiki/Singular_value_decomposition" rel="nofollow">http://en.wikipedia.org/wiki/Singular_value_decomposition</a>). There is a very thorough discussion, including a section "Geometric meaning" in which your interpretation is clearly explained. Well, there $K^n$, where $K = \mathbb R$ or $K = \mathbb C$ is used, instead of arbitrary inner product spaces, but I suppose you'll agree that the difference is not essential.</p> http://mathoverflow.net/questions/43408/is-this-formulation-of-the-singular-value-decomposition-standard/43427#43427 Answer by Ramsay for Is this formulation of the Singular Value Decomposition standard? Ramsay 2010-10-24T20:37:23Z 2011-01-07T13:43:46Z <p>The book "Numerical Linear Algebra" by Trefethen and Bau, also introduces the SVD in this way. The relevant chapters (4 and 5) seem to be complete in Google books. The exposition is not tainted by the 'numerical' in the title. </p> <p>I don't understand why the SVN isn't given more emphasis in standard introductions to linear algebra. It seems to give a good intuitive decomposition of a linear operator. I would expect it to be a useful theoretical tool even if numerical issues are not being considered.</p> http://mathoverflow.net/questions/43408/is-this-formulation-of-the-singular-value-decomposition-standard/45541#45541 Answer by Denis Serre for Is this formulation of the Singular Value Decomposition standard? Denis Serre 2010-11-10T09:59:24Z 2010-11-10T15:10:38Z <p>The singular values do have a sound geometrical meaning. The first one is nothing but the operator norm of $T$, that is the maximum dilation coefficient $$\frac{\|Tx\|}{\|x\|}.$$ The next ones can be seen also as maximum of dilation coefficients, provided you replace lines by subspaces. For instance $$s_k=\max_{\dim F=k}\min\left\{\frac{\|Tx\|}{\|x\|};x\in F,x\ne0\right\}.$$ In terms of the exterior algebra over $\mathbb C^k$ ($k=m$ or $n$), you have $$s_1\cdots s_k=\sup\left\{\frac{\|Tx_1\wedge\cdots\wedge Tx_k\|}{\|x_1\wedge\cdots\wedge x_k\|};x_1,\ldots,x_k\in F\right\}.$$</p> <p>Notice also that there is a $p$-adic version of the SVD decomposition. See K. S. Kedlaya. <em>$p$-adic differential equations</em>. Cambridge Studies in Advanced Mathematics, 125. Cambridge University Press, Cambridge, 2010</p> <p><strong>Edit</strong>. I should have also given the formula $$s_1+\cdots+s_k=\max\{{\rm Tr}(PTQ); P\hbox{ and } Q \hbox{ are unitary projectors of rank } k\}.$$ This has the interesting consequence that $T\mapsto s_1+\cdots+s_k$ is a convex function.</p> <p><strong>Edit (bis)</strong>. Actually, $T\mapsto s_1\cdots s_k$ is <em>rank-one convex</em>. This means that it is convex along every line $A+{\mathbb R} B$ for which $B-A$ is a rank-one matrix.</p> http://mathoverflow.net/questions/43408/is-this-formulation-of-the-singular-value-decomposition-standard/45573#45573 Answer by wildildildlife for Is this formulation of the Singular Value Decomposition standard? wildildildlife 2010-11-10T17:21:21Z 2010-11-10T17:21:21Z <p>It's in my favourite linear algebra book <a href="http://www.amazon.com/Advanced-Linear-Algebra-Graduate-Mathematics/dp/0387728287" rel="nofollow">Advanced Linear Algebra by Steven Roman</a>, chapter 17 (called Singular Values and the Moore-Penrose inverse).</p>