3
$\begingroup$

Let $A$ be a $m\times n$ real matrix, whose entries are independent, identically distributed random variables, following standard normal distributions (mean zero and unit variance).

What is the distribution of the singular values and singular (left and right) vectors of $A$?

For a symmetric square matrix, the concept of rotational invariance is helpful because (for ensembles where it holds) it allows us to focus on the eigenvalues since the eigenvectors are essentially uniformly distributed.

What is the analogue of "rotational invariance" in the context of a rectangular matrix like $A$? Is it helfpul here too?

Most papers/books I've read on the topic focus on square matrices so if anyone can point out relevant references I'd also appreciate that.

$\endgroup$

1 Answer 1

5
$\begingroup$

To be specific, let me assume $m\leq n$. The $m\times n$ matrix $A$ then has $n-m$ singular values equal to zero. The remaining $m$ singular values $s_i$, $i=1,2,\ldots m$ are the positive square roots of the eigenvalues $\sigma_i$ of the symmetric matrix product $AA^t$. The distribution $P(\sigma_1,\sigma_2,\ldots\sigma_m)$ of the $\sigma_i$'s is the Wishart distribution. For $m\gg 1$ the eigenvalue density is the Marchenko–Pastur distribution.

The eigenvectors of $AA^t$ are the left eigenvectors in the singular value decomposition. These are uniformly distributed with the Haar measure in $O(m)$. The right eigenvectors are $m$ rows of length $n$ of an $n\times n$ matrix that is uniformly distributed in $O(n)$, independently of the left eigenvectors.

$\endgroup$
8
  • $\begingroup$ Thanks! But how can you tell that the right singular vectors are independent from the left ones? Also, which of these statements rely upon the components of $A$ being Gaussian? I presume that for a non-Gaussian distribution, the left and right vectors become relevant (because the rotational invariance is broken in some sense). $\endgroup$
    – valle
    Feb 9, 2020 at 15:58
  • 1
    $\begingroup$ your distribution $P(A)\propto\exp(-\tfrac{1}{2}\,{\rm tr}\,AA^t)$ is both left-invariant and right-invariant under multiplication by an arbitrary orthogonal matrix, $P(AO)=P(O'A)$, so the left- and right-eigenvectors can be chosen independently and uniformly; all of this is for a Gaussian distribution. $\endgroup$ Feb 9, 2020 at 20:04
  • $\begingroup$ I see now, thanks! $\endgroup$
    – valle
    Feb 9, 2020 at 20:07
  • $\begingroup$ What is the analogue of the Vandermonde determinant, if I want to change variables using a singular value decomposition? (Please see mathoverflow.net/questions/352309/…) $\endgroup$
    – valle
    Feb 9, 2020 at 23:02
  • 1
    $\begingroup$ Forrester is highly recommended: contains detailed derivations. $\endgroup$ Feb 10, 2020 at 7:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.