Consider a random matrix $A \in \mathbb{R}^{m\times n}$ with i.i.d. entries, with mean zero and variance 1 and $m <n $. I am interested in the expectation of $$E_{A}(\mathrm{Tr}( (A^T A + \lambda \mathrm{Id})^{1})).$$ This question is similar to Trace of inverse of random positivedefinite matrix in high dimension?, except $A$ is nonsymmetric. I am unfamiliar with MarchenkoPastur distribution, so any references would be great!
1 Answer
I think the product $A^\top A$ in the OP should read $AA^\top$, to avoid a trivial contribution from zero eigenvalues (assuming $A\in\mathbb{R}^{m\times n}$ and $m<n$).
For $m,n\gg 1$, and $m/n\equiv r\in (0,1)$ fixed, an integration over the Marchenko–Pastur distribution gives the answer (with $x_\pm=(1\pm\sqrt{r})^2$) $$\lim_{m,n\rightarrow\infty}\mathbb{E}[m^{1}\mathrm{Tr}\,(n^{1}AA^\top + \lambda I)^{1})]=\int_{x_}^{x_+} \frac{1}{x+\lambda}\frac{\sqrt{\left(x_+x\right) \left(xx_\right)}}{2 \pi r x}\,dx$$ $$\qquad=\frac{1}{2\lambda r}\left(\sqrt{\lambda^2+2 \lambda (1+r)+(1r)^2}\lambda+r1\right).$$ The rescaling of $AA^\top$ by a factor $1/n$ is needed for a $\lambda$dependent answer in the large $n$ limit. If you do not rescale, then $$\lim_{m,n\rightarrow\infty}\mathbb{E}[\mathrm{Tr}\,(AA^\top + \lambda I)^{1})]=\frac{r}{1r}.$$ This diverges for $r=1$, in that case the trace grows as $\sqrt{n}$, see https://mathoverflow.net/a/332889/11260

$\begingroup$ Could you show the integration bounds? I.e more explicitly how the integration is calculated. In addition, how would the calculation change if we had a diagonal matrix $D \in \mathbb{R}^{n\times n}$ $$E_{A}(\mathrm{Tr}( (DA^T AD + \lambda \mathrm{Id})^{1})),$$ so you are unable to exchange the transposes on A. $\endgroup$– gokuCommented Jun 15, 2022 at 18:34

$\begingroup$ I added the integral formula; if you insert a matrix $D$, even if it is diagonal, the answer is no longer only a function of the eigenvalues of $AA^\top$, so the MP distribution does not help; this has nothing to do with the order $AA^\top$ or $A^\top A$, its whether the eigenvectors enter in addition to the eigenvalues. $\endgroup$ Commented Jun 15, 2022 at 18:43
