Take the 2-minute tour ×
MathOverflow is a question and answer site for professional mathematicians. It's 100% free, no registration required.

Given a $N \times M$ matrix $X$ comprised of standard normal entries ($M > N$), I'm interested in approximating $E[trace((XX^T\frac{\gamma}{M} + I)^{-1}]$ in terms of $N, M$ and $\gamma$. Unfortunately, I can't necessarily assume $\gamma$ is small. I've had no luck in coming up with any kind of approximation. Thanks!

For context, this problem relates to the effective degrees of freedom for ridge regression.

share|improve this question

1 Answer 1

up vote 3 down vote accepted

let $\lambda_{1},\lambda_2,\ldots\lambda_N$ be the eigenvalues of $M^{-1}XX^{T}$; including for convenience a factor $1/N$, the quantity you seek is

$$N^{-1}E[{\rm Tr}(XX^{T}\gamma/M)+I)^{-1}]=\int d\lambda \rho(\lambda)(\lambda\gamma+1)^{-1}$$

where $\rho(\lambda)=E[N^{-1}\sum_n\delta(\lambda-\lambda_n)]$ is the eigenvalue density of Wishart matrices; this quantity is known in closed form for $N,M\rightarrow\infty$ at finite ratio $N/M=r\in(0,1]$ (Marchenko-Pastur distribution). I find in this way the answer

$$\lim_{N,M\rightarrow\infty}N^{-1}E[{\rm Tr}(XX^{T}\gamma/M)+I)^{-1}]=(2r\gamma)^{-1}\left(-1-\gamma\sqrt{ab}+\sqrt{(1+a\gamma)(1+b\gamma)}\right)$$

with $a=(1+\sqrt r)^2$ and $b=(1-\sqrt r)^2$. As a check, you can take the limit $\gamma\rightarrow 0$ of this expression and obtain $1$, as it should.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.