1
$\begingroup$

Let $x_1,\dots,x_n$ be i.i.d. $N(0,I_{p\times p})$, with $n>p$. Let $\hat S=\frac1n\sum_{i=1}^n x_i x_i^T$ be the sample covariance.

Assume the asymptotic setting where $\frac pn\to \alpha<1$.

Is there a result about the concentration of $\mathbb{E}\left[\operatorname{tr}\left(\hat{S}^{-1}\right)\right]$ in the asymptotic setting? By using Lemma 3.2 in this paper coupled with the Sherman-Morrison formula I believe that $\mathbb{E}\left[\operatorname{tr}\left(\hat{S}^{-1}\right)\right]=\frac{1}{1-\alpha}$, but I haven't been able to make it rigorous.

Are there any existing results about this?

$\endgroup$

2 Answers 2

1
$\begingroup$

Writing $\hat{S}= \frac1n XX^T$ where $X$ has columns $x_1,\dots,x_n$, we can express $$ \text{tr}(\hat{S}^{-1}) = \sum_{i=1}^p \lambda_i(\hat{S})^{-1} = n\sum_{i=1}^p \sigma_i(X)^{-2} $$ where $\sigma_1(X), \dots, \sigma_p(X)$ are the nonzero singular values of $X$. Then we can use the inverse second moment identity (observed by Tao and Vu here http://arxiv.org/pdf/0807.4898v5.pdf; the rectangular case is given as Lemma 4.14 here: https://arxiv.org/pdf/1109.3343v4.pdf) to express $$\sum_{i=1}^p \sigma_i(X)^{-2}=\sum_{i=1}^p \text{dist}(R_i,R_{-i})^{-2}$$ where $\text{dist}(R_i,R_{-i})$ is the distance from the $i$th row of $X$ to the span of the remaining rows. Since $X$ has normally distributed entries the variables $\text{dist}(R_i,R_{-i})^{2}$ have chi-squared distribution, so the expectation of this sum can be computed explicitly. Unless I have made some mistake it comes out to $\mathbb{E}\frac1p\text{tr}(\hat{S}^{-1}) = \frac{n}{n-p-1} = \frac{1}{1-\alpha}+o(1)$ (note we need the normalization $1/p$).

$\endgroup$
1
  • $\begingroup$ Thanks. I was indeed able to use the lemma I had cited with the method I sketched, but it didn't yield the expectation for the finite n and p case. Thanks for the nice references! $\endgroup$ Jul 21, 2016 at 22:44
0
$\begingroup$

The formula $\mathbb E[(\sum_{i=1}^n x_ix_i^T)^{-1}] = I_p/(n-p-1)$ is exact. It is known on Wikipedia as the expectation of the inverse wishart distribtion: https://en.wikipedia.org/wiki/Inverse-Wishart_distribution

To prove it, outside of the diagonal for some $i\ne j$: with $X$ the matrix with rows $x_1,...,x_n$ and $e_j$ the $j$-th canonical vector, the fact that $X(I_p - 2e_je_j^T)$ has the same distribution as $X$ (i.e., changing the sign of the $j$-th column) gives that the off-diagional $i,j$-th element is 0.

For the diagonal elements, by symmetry it is enough to compute $trace[(X^TX)^{-1}]$ as asked in the question. By Stein's formula $\mathbb E[x_i^T F(X)]=\sum_j \mathbb E[ e_j^T \frac{\partial}{\partial x_{ij}} F(X)]$ for $F$ valued in $R^p$ we find \begin{align} \mathbb E[x_i^T(X^TX)^{-1}x_i] =\mathbb E\Big[\sum_{j=1}^p e_j^T(X^TX)^{-1} e_j - e_j^T(X^TX)^{-1}(e_j x_i^T + x_i e_j^T)(X^TX)^{-1} x_i\Big]. \end{align} Summing over $i=1,...,n$ $$\mathbb E\sum_{i=1}^n x_i^T(X^TX)^{-1}x_i = n \mathbb E\Big[Tr[(X^TX)^{-1}] - Tr[(X^TX)^{-1}]\sum_{i=1}^n x_i^T(X^TX)^{-1}x_i - Tr[(X^TX)^{-1}\sum_{i=1}^nx_ix_i^T(X^TX)^{-1}] \Big] $$ or equivalently $p=(n-p-1)\mathbb E Tr[(X^TX)^{-1}]$ which is the desired identity.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.