Given a $N \times M$ matrix $X$ comprised of standard normal entries ($M > N$), I'm interested in approximating $E[trace((XX^T\frac{\gamma}{M} + I)^{-1}]$ in terms of $N, M$ and $\gamma$. Unfortunately, I can't necessarily assume $\gamma$ is small. I've had no luck in coming up with any kind of approximation. Thanks!
For context, this problem relates to the effective degrees of freedom for ridge regression.
let $\lambda_{1},\lambda_2,\ldots\lambda_N$ be the eigenvalues of $M^{-1}XX^{T}$; including for convenience a factor $1/N$, the quantity you seek is
$$N^{-1}E[{\rm Tr}(XX^{T}\gamma/M)+I)^{-1}]=\int d\lambda \rho(\lambda)(\lambda\gamma+1)^{-1}$$
where $\rho(\lambda)=E[N^{-1}\sum_n\delta(\lambda-\lambda_n)]$ is the eigenvalue density of Wishart matrices; this quantity is known in closed form for $N,M\rightarrow\infty$ at finite ratio $N/M=r\in(0,1]$ (Marchenko-Pastur distribution). I find in this way the answer
$$\lim_{N,M\rightarrow\infty}N^{-1}E[{\rm Tr}(XX^{T}\gamma/M)+I)^{-1}]=(2r\gamma)^{-1}\left(-1-\gamma\sqrt{ab}+\sqrt{(1+a\gamma)(1+b\gamma)}\right)$$
with $a=(1+\sqrt r)^2$ and $b=(1-\sqrt r)^2$. As a check, you can take the limit $\gamma\rightarrow 0$ of this expression and obtain $1$, as it should.
