Skip to main content
1 of 3
jlewk
  • 1.7k
  • 7
  • 14

If $R$ has iid entries with $k$ rows and $d$ columns ($k<d$), then by rotational invariance the SVD $R=UDV^T$ satisfies

  • $U\in O(k)$ is uniformly distributed (Haar measure on $O(k)$,
  • the diagonal matrix $D\in R^k$ contains the random singularvalues,
  • $V\in R^{d\times k}$ has $k$ orthonormal columns and is distributed according to the Grassmanian.

And $(U, D, V)$ are independent.

The pseudo-inverse is $R^+ = VD^{-1} U^T$. Looking at one entry specifically, say we multiply to the left by $x\in S^{d-1}$ and to the right by $y\in S^{k-1}$, then $\hat y = U y \in S^{k-1}$ is uniformly distributed on the sphere and so is $\hat x = V^Tx \in S^{k-1}$, and the random variable of interest is $Z = \hat x^T (D^{-1}) \hat y.$ Writing $\hat x = g/\|g\|$ for standard normal $g$, $\|g\|^2\sim \chi^2_k$ and the above random variable is equal in distirbution to $$ N(0,1) \cdot \|D^{-1} \hat y\|/ \sqrt{\chi^2_k} $$ If $d,k$ are such that $\|D^{-1} \hat y\|^2\approx {trace[D^{-2}]}/k$ by, e.g., the Hanson-Wright concentration inequality, and if $trace[D^{-2}]=trace[(RR^T)^{-1}]$ converges to its expectation $\frac{k}{d-k-1}$ (https://en.wikipedia.org/wiki/Inverse-Wishart_distribution), the asymptotic variance can be characterized.

jlewk
  • 1.7k
  • 7
  • 14