Skip to main content
1 of 5
Carlo Beenakker
  • 188.2k
  • 18
  • 448
  • 651

Let me calculate the expectation value of $\alpha$. The probability distribution of $X$ is invariant under orthogonal transformations, so without loss of generality I can orient the unit vector $w$ along one of the axes, $w_i=\delta_{ip}$, $p\in\{1,2,\ldots d\}$. Then $$\mathbb{E}[\alpha]=\frac{1}{d}\mathbb{E}\left(X^T(XX^T+\lambda I)^{-1}XX^T(XX^T+\lambda I)^{-1}X\right)_{pp}.$$ Again because of orthogonal invariance the answer cannot depend on the value of the index $p$, hence we can sum over $p$ and divide by $d$, which gives the trace, $$\mathbb{E}[\alpha]=\frac{1}{d^2}\mathbb{E}\,{\rm tr}\,\left(X^T(XX^T+\lambda I)^{-1}XX^T(XX^T+\lambda I)^{-1}X\right)_{pp}$$ $$\qquad=\frac{1}{d^2}\mathbb{E}\,{\rm tr}\,\frac{W^2}{(W+\lambda I)^2},\;\;W=XX^{T}.$$

For the subcase $\lambda\rightarrow 0$ in the OP we thus find $\mathbb{E}[\alpha]=1/d$ for any $n$. (No need to take the large-$n$ limit.)

For nonzero $\lambda$ and in the large-$n$ limit the result for $\mathbb{E}[\alpha]$ is the integral of $\mu^2(\mu+\lambda)^{-2}$ weighted by the Marchenko-Pastur distribution $\rho(\mu)$ for the eigenvalues $\mu$ of $W$.

Carlo Beenakker
  • 188.2k
  • 18
  • 448
  • 651