1
$\begingroup$

Let $x_1,\dots,x_n$ be i.i.d. drawn from $N(0,I_{p\times p})$. Let $\hat S = \frac 1n \sum_{i=1}^n x_ix_i^T$ be the sample covariance. Let $\lambda_1 \ge ... \ge \lambda_p$ be the eigenvalues of $\hat S$.

It is known that $$\mathbb E\frac1p\operatorname{tr}\left(\hat S^{-1}\right) = \mathbb E\frac1p\sum_{i=1}^p \frac{1}{\lambda_i} = \frac{n}{n-p-1},$$ Therefore, if $p=n$ this quantity tends to $\infty$ as $n,p\to \infty$.

Suppose that $\beta>0$ and consider the sum $$\frac1p\sum_{i=1}^p \frac{1}{\lambda_i+\beta}.$$

Given any number $a>0$, is there a $\beta$ such that as $n\to\infty $ and $p/n\to 1$ $$\mathbb E\frac1p\sum_{i=1}^p \frac{1}{\lambda_i+\beta}\to a?$$ This looks like it should be straightforward, but the analysis appears subtle because of the behavior of the least eigenvalues of $\hat S$. It doesn't seem quite enough to show that for any $\beta>0$ this sum is positive and monotone increasing, because of the question of how the eigenvalues behave in the limit.

Any help would be greatly appreciated.

$\endgroup$
  • $\begingroup$ Actually this seems to be not too bad since the sum in question converges to the integral of $1/(x+\beta)$ against the Marcenko-Pastur distribution. $\endgroup$ – Lepidopterist Oct 19 '16 at 19:24
1
$\begingroup$

The answer is yes. Indeed, by the Marcenko-Pastur Theorem, we have that $$ \frac{1}{p} \sum_{i=1}^p \frac{1}{\lambda_i + \beta} \overset{d}{\to} \int_0^4 \frac{1}{x+\beta} \frac{\sqrt{(4-x) x}}{2 \pi x} dx \quad \text{as $n \to \infty$ and $\frac{p}{n} \to 1$} $$ Note that this limit is positive for all $\beta>0$, strictly decreasing with $\beta$, and $\infty$ at $\beta=0$.

$\endgroup$
  • $\begingroup$ Yes, this is what I said in my comment above. $\endgroup$ – Lepidopterist Oct 19 '16 at 21:24
  • $\begingroup$ Based on your comment, it was unclear to me that this question got resolved. $\endgroup$ – Nawaf Bou-Rabee Oct 19 '16 at 22:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.