# Could SVD be used to optimize the partial inner-products?

Suppose a set $N$ of $n$ distinct points in $m-$dimensional space is given in $X\in\mathbb{R}^{n\times m}$. Also, suppose a subset $L\subset N$, $|L|=l<m<n$, with $m-$dimensional coordinates in $X_l\in\mathbb{R}^{l\times m}$.

Now, given a inner-product matrix $M=XX_l^T\in\mathbb{R}^{n\times l}$, could the solution $Z$ minimizing $$||M-ZZ_l^T||~~~\equiv~~~||M-ZZ_l^T||^2,$$ where $||\cdot||$ denotes the Frobenius matrix norm, over all $Z\in\mathbb{R}^{n\times k}$, $k<m$, be obtained in a closed form. Note that $Z_l\in\mathbb{R}^{l\times k}$ is a configuration corresponding to the subset $L\subset N$ defined above.

With the Eckart-Young theorem in mind, I'm inclined to think that the solution $Z$ might be $$Z=U\Sigma_k^{1/2},$$ where $\Sigma_k^{1/2}$ is a diagonal matrix containing square roots of only $k$ dominant singular values of $M$, with $U$ containing the corresponding left-singular vectors as its columns. The plain Eckart-Young theorem might be used by showing that $Z_l$ is related to right-singular vectors $V$, as in $$Z_l=V\Sigma_k^{1/2},$$ but I believe the answer might not be easily expressible in a closed-form. Feel free to post an existing theorem that might be helpful for proving the above, either exactly or with certain guarantees on the approximation, ie. "by taking $$Z=U\Sigma_k^{1/2},$$ as above, we are guaranteed that the error wrt $$\min_Z||M-ZZ_l^T||~~~\equiv~~~\min_Z||M-ZZ_l^T||^2$$ is not larger than ..." Also, notes on some related problems already present in the literature will be appreciated.

Note that in case $L=N$, the solution indeed translates to the Eckart-Young theorem, thence with optimal solution being based (now) on spectral decomposition of $M$ relying only on its $k$ dominant eigenvalues, hence with $$Z=E\Lambda_k^{1/2},$$ where $\Lambda_k^{1/2}$ is a diagonal matrix with square roots of only $k$ dominant eigenvalues of $M$, and $E$ is a matrix containing the corresponding eigenvalues.

-
Could someone edit my question? The preview seems not to capture everything I wrote. – usero Mar 14 '12 at 16:25
Done. As a rule of thumb, put backticks (...) around formulas when they are displayed incorrectly (as is reminded in the "How to write math" box on the right of every page). – Federico Poloni Mar 14 '12 at 16:30