MathOverflow is a question and answer site for professional mathematicians. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Given a $N \times M$ matrix $X$ comprised of standard normal entries ($M > N$), I'm interested in approximating $E[trace((XX^T\frac{\gamma}{M} + I)^{-1}]$ in terms of $N, M$ and $\gamma$. Unfortunately, I can't necessarily assume $\gamma$ is small. I've had no luck in coming up with any kind of approximation. Thanks!

For context, this problem relates to the effective degrees of freedom for ridge regression.

share|cite|improve this question
up vote 3 down vote accepted

let $\lambda_{1},\lambda_2,\ldots\lambda_N$ be the eigenvalues of $M^{-1}XX^{T}$; including for convenience a factor $1/N$, the quantity you seek is

$$N^{-1}E[{\rm Tr}(XX^{T}\gamma/M)+I)^{-1}]=\int d\lambda \rho(\lambda)(\lambda\gamma+1)^{-1}$$

where $\rho(\lambda)=E[N^{-1}\sum_n\delta(\lambda-\lambda_n)]$ is the eigenvalue density of Wishart matrices; this quantity is known in closed form for $N,M\rightarrow\infty$ at finite ratio $N/M=r\in(0,1]$ (Marchenko-Pastur distribution). I find in this way the answer

$$\lim_{N,M\rightarrow\infty}N^{-1}E[{\rm Tr}(XX^{T}\gamma/M)+I)^{-1}]=(2r\gamma)^{-1}\left(-1-\gamma\sqrt{ab}+\sqrt{(1+a\gamma)(1+b\gamma)}\right)$$

with $a=(1+\sqrt r)^2$ and $b=(1-\sqrt r)^2$. As a check, you can take the limit $\gamma\rightarrow 0$ of this expression and obtain $1$, as it should.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.