# How to control the average eigenvalue of a “regularized” inverse sample covariance?

Let $x_1,\dots,x_n$ be i.i.d. drawn from $N(0,I_{p\times p})$. Let $\hat S = \frac 1n \sum_{i=1}^n x_ix_i^T$ be the sample covariance. Let $\lambda_1 \ge ... \ge \lambda_p$ be the eigenvalues of $\hat S$.

It is known that $$\mathbb E\frac1p\operatorname{tr}\left(\hat S^{-1}\right) = \mathbb E\frac1p\sum_{i=1}^p \frac{1}{\lambda_i} = \frac{n}{n-p-1},$$ Therefore, if $p=n$ this quantity tends to $\infty$ as $n,p\to \infty$.

Suppose that $\beta>0$ and consider the sum $$\frac1p\sum_{i=1}^p \frac{1}{\lambda_i+\beta}.$$

Given any number $a>0$, is there a $\beta$ such that as $n\to\infty$ and $p/n\to 1$ $$\mathbb E\frac1p\sum_{i=1}^p \frac{1}{\lambda_i+\beta}\to a?$$ This looks like it should be straightforward, but the analysis appears subtle because of the behavior of the least eigenvalues of $\hat S$. It doesn't seem quite enough to show that for any $\beta>0$ this sum is positive and monotone increasing, because of the question of how the eigenvalues behave in the limit.

Any help would be greatly appreciated.

• Actually this seems to be not too bad since the sum in question converges to the integral of $1/(x+\beta)$ against the Marcenko-Pastur distribution. – Lepidopterist Oct 19 '16 at 19:24

## 1 Answer

The answer is yes. Indeed, by the Marcenko-Pastur Theorem, we have that $$\frac{1}{p} \sum_{i=1}^p \frac{1}{\lambda_i + \beta} \overset{d}{\to} \int_0^4 \frac{1}{x+\beta} \frac{\sqrt{(4-x) x}}{2 \pi x} dx \quad \text{as n \to \infty and \frac{p}{n} \to 1}$$ Note that this limit is positive for all $\beta>0$, strictly decreasing with $\beta$, and $\infty$ at $\beta=0$.

• Yes, this is what I said in my comment above. – Lepidopterist Oct 19 '16 at 21:24
• Based on your comment, it was unclear to me that this question got resolved. – Nawaf Bou-Rabee Oct 19 '16 at 22:34