# Asymptotics of $w^\top G^2 w$, where $w$ is a unit-vector, $G:=X^T(XX^T+t I_n)^{-1}X$, $t > 0$, and $X$ is an $n\times d$ gaussian random matrix

Let $$X$$ be an random $$n \times d$$ matrix with entries drawn iid from $$N(0,1/d)$$ and let $$w$$ be a unit-vector in $$\mathbb R^d$$. With $$\lambda>0$$, and define $$G:=X^\top(XX^\top + \lambda I_n)^{-1}X$$. Finally, defined $$\alpha := w^\top G^2 w$$.

Question. In the limit $$n,d \to \infty$$ with $$n/d \to \rho \in (0,\infty)$$, what is the limitting value of $$\alpha$$ as a function of $$\lambda$$ and $$\rho$$ ?

A useful subcase is when $$\lambda \to 0^+$$.

Question. What is the value of $$\lim_{\lambda \to 0^+}\lim_{n,d \to \infty \\ n/d \to \rho}\alpha$$ as a function of $$\rho$$ ?

Let me calculate the expectation value of $$\alpha$$. The probability distribution of $$X$$ is invariant under orthogonal transformations, so without loss of generality I can orient the unit vector $$w$$ along one of the axes, $$w_i=\delta_{ip}$$, $$p\in\{1,2,\ldots d\}$$. Then $$\mathbb{E}[\alpha]=\mathbb{E}\left(X^T(XX^T+\lambda I)^{-1}XX^T(XX^T+\lambda I)^{-1}X\right)_{pp}.$$ Again because of orthogonal invariance the answer cannot depend on the value of the index $$p$$, hence we can sum over $$p$$ and divide by $$d$$, which gives the trace, $$\mathbb{E}[\alpha]=\frac{1}{d}\mathbb{E}\,{\rm tr}\,\left(X^T(XX^T+\lambda I)^{-1}XX^T(XX^T+\lambda I)^{-1}X\right)$$ $$\qquad=\frac{1}{d}\mathbb{E}\,{\rm tr}\,\frac{W^2}{(W+\lambda I)^2},\;\;W=XX^{T}.$$
For the subcase $$\lambda\rightarrow 0$$ in the OP we thus find $$\mathbb{E}[\alpha]=n/d \to \rho$$.
For nonzero $$\lambda$$ and in the large-$$n$$ limit the result for $$\mathbb{E}[\alpha]$$ is the integral of $$\mu^2(\mu+\lambda)^{-2}$$ weighted by the Marchenko-Pastur distribution $$\rho(\mu)$$ for the eigenvalues $$\mu$$ of $$W$$.
• Thanks. I should've thought of rotational-invariance! BTW, it the case $\lambda \to 0^+$, shouldn't one have $\mathbb E[\alpha] = \rho$ instead ? Oct 1, 2021 at 20:45
• yes, $\rho$, corrected. Oct 1, 2021 at 20:46