1
$\begingroup$

Let $x_1,\dots,x_n$ be i.i.d. $N(0,I_{p\times p})$, with $n>p$. Let $\hat S=\frac1n\sum_{i=1}^n x_i x_i^T$ be the sample covariance.

Assume the asymptotic setting where $\frac pn\to \alpha<1$.

Is there a result about the concentration of $\mathbb{E}\left[\operatorname{tr}\left(\hat{S}^{-1}\right)\right]$ in the asymptotic setting? By using Lemma 3.2 in this paper coupled with the Sherman-Morrison formula I believe that $\mathbb{E}\left[\operatorname{tr}\left(\hat{S}^{-1}\right)\right]=\frac{1}{1-\alpha}$, but I haven't been able to make it rigorous.

Are there any existing results about this?

$\endgroup$
1
$\begingroup$

Writing $\hat{S}= \frac1n XX^T$ where $X$ has columns $x_1,\dots,x_n$, we can express $$ \text{tr}(\hat{S}^{-1}) = \sum_{i=1}^p \lambda_i(\hat{S})^{-1} = n\sum_{i=1}^p \sigma_i(X)^{-2} $$ where $\sigma_1(X), \dots, \sigma_p(X)$ are the nonzero singular values of $X$. Then we can use the inverse second moment identity (observed by Tao and Vu here http://arxiv.org/pdf/0807.4898v5.pdf; the rectangular case is given as Lemma 4.14 here: https://arxiv.org/pdf/1109.3343v4.pdf) to express $$\sum_{i=1}^p \sigma_i(X)^{-2}=\sum_{i=1}^p \text{dist}(R_i,R_{-i})^{-2}$$ where $\text{dist}(R_i,R_{-i})$ is the distance from the $i$th row of $X$ to the span of the remaining rows. Since $X$ has normally distributed entries the variables $\text{dist}(R_i,R_{-i})^{2}$ have chi-squared distribution, so the expectation of this sum can be computed explicitly. Unless I have made some mistake it comes out to $\mathbb{E}\frac1p\text{tr}(\hat{S}^{-1}) = \frac{n}{n-p-1} = \frac{1}{1-\alpha}+o(1)$ (note we need the normalization $1/p$).

$\endgroup$
  • $\begingroup$ Thanks. I was indeed able to use the lemma I had cited with the method I sketched, but it didn't yield the expectation for the finite n and p case. Thanks for the nice references! $\endgroup$ – Lepidopterist Jul 21 '16 at 22:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.