Take the 2-minute tour ×
MathOverflow is a question and answer site for professional mathematicians. It's 100% free, no registration required.

Given a $N \times M$ matrix $X$ comprised of standard normal entries ($M > N$), I'm interested in approximating $E[trace((XX^T\frac{\gamma}{M} + I)^{-1}]$ in terms of $N, M$ and $\gamma$. Unfortunately, I can't necessarily assume $\gamma$ is small. I've had no luck in coming up with any kind of approximation. Thanks!

For context, this problem relates to the effective degrees of freedom for ridge regression.

share|improve this question
add comment

1 Answer

up vote 3 down vote accepted

let $\lambda_{1},\lambda_2,\ldots\lambda_N$ be the eigenvalues of $M^{-1}XX^{T}$; including for convenience a factor $1/N$, the quantity you seek is

$$N^{-1}E[{\rm Tr}(XX^{T}\gamma/M)+I)^{-1}]=\int d\lambda \rho(\lambda)(\lambda\gamma+1)^{-1}$$

where $\rho(\lambda)=E[N^{-1}\sum_n\delta(\lambda-\lambda_n)]$ is the eigenvalue density of Wishart matrices; this quantity is known in closed form for $N,M\rightarrow\infty$ at finite ratio $N/M=r\in(0,1]$ (Marchenko-Pastur distribution). I find in this way the answer

$$\lim_{N,M\rightarrow\infty}N^{-1}E[{\rm Tr}(XX^{T}\gamma/M)+I)^{-1}]=(2r\gamma)^{-1}\left(-1-\gamma\sqrt{ab}+\sqrt{(1+a\gamma)(1+b\gamma)}\right)$$

with $a=(1+\sqrt r)^2$ and $b=(1-\sqrt r)^2$. As a check, you can take the limit $\gamma\rightarrow 0$ of this expression and obtain $1$, as it should.

share|improve this answer
add comment

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.