2
$\begingroup$

Consider a random matrix $A \in \mathbb{R}^{m\times n}$ with i.i.d. entries, with mean zero and variance 1 and $m <n $. I am interested in the expectation of $$E_{A}(\mathrm{Tr}( (A^T A + \lambda \mathrm{Id})^{-1})).$$ This question is similar to Trace of inverse of random positive-definite matrix in high dimension?, except $A$ is non-symmetric. I am unfamiliar with Marchenko-Pastur distribution, so any references would be great!

$\endgroup$

1 Answer 1

1
$\begingroup$

I think the product $A^\top A$ in the OP should read $AA^\top$, to avoid a trivial contribution from zero eigenvalues (assuming $A\in\mathbb{R}^{m\times n}$ and $m<n$).

For $m,n\gg 1$, and $m/n\equiv r\in (0,1)$ fixed, an integration over the Marchenko–Pastur distribution gives the answer (with $x_\pm=(1\pm\sqrt{r})^2$) $$\lim_{m,n\rightarrow\infty}\mathbb{E}[m^{-1}\mathrm{Tr}\,(n^{-1}AA^\top + \lambda I)^{-1})]=\int_{x_-}^{x_+} \frac{1}{x+\lambda}\frac{\sqrt{\left(x_+-x\right) \left(x-x_-\right)}}{2 \pi r x}\,dx$$ $$\qquad=\frac{1}{2\lambda r}\left(\sqrt{\lambda^2+2 \lambda (1+r)+(1-r)^2}-\lambda+r-1\right).$$ The rescaling of $AA^\top$ by a factor $1/n$ is needed for a $\lambda$-dependent answer in the large $n$ limit. If you do not rescale, then $$\lim_{m,n\rightarrow\infty}\mathbb{E}[\mathrm{Tr}\,(AA^\top + \lambda I)^{-1})]=\frac{r}{1-r}.$$ This diverges for $r=1$, in that case the trace grows as $\sqrt{n}$, see https://mathoverflow.net/a/332889/11260

$\endgroup$
3
  • $\begingroup$ Could you show the integration bounds? I.e more explicitly how the integration is calculated. In addition, how would the calculation change if we had a diagonal matrix $D \in \mathbb{R}^{n\times n}$ $$E_{A}(\mathrm{Tr}( (DA^T AD + \lambda \mathrm{Id})^{-1})),$$ so you are unable to exchange the transposes on A. $\endgroup$
    – goku
    Commented Jun 15, 2022 at 18:34
  • $\begingroup$ I added the integral formula; if you insert a matrix $D$, even if it is diagonal, the answer is no longer only a function of the eigenvalues of $AA^\top$, so the MP distribution does not help; this has nothing to do with the order $AA^\top$ or $A^\top A$, its whether the eigenvectors enter in addition to the eigenvalues. $\endgroup$ Commented Jun 15, 2022 at 18:43
  • $\begingroup$ thank you very much! $\endgroup$
    – goku
    Commented Jun 15, 2022 at 18:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.