# Trace of the inverse sample covariance as the number of samples and dimension scale to infinity

Let $x_1,\dots,x_n$ be i.i.d. $N(0,I_{p\times p})$, with $n>p$. Let $\hat S=\frac1n\sum_{i=1}^n x_i x_i^T$ be the sample covariance.

Assume the asymptotic setting where $\frac pn\to \alpha<1$.

Is there a result about the concentration of $\mathbb{E}\left[\operatorname{tr}\left(\hat{S}^{-1}\right)\right]$ in the asymptotic setting? By using Lemma 3.2 in this paper coupled with the Sherman-Morrison formula I believe that $\mathbb{E}\left[\operatorname{tr}\left(\hat{S}^{-1}\right)\right]=\frac{1}{1-\alpha}$, but I haven't been able to make it rigorous.

Writing $\hat{S}= \frac1n XX^T$ where $X$ has columns $x_1,\dots,x_n$, we can express $$\text{tr}(\hat{S}^{-1}) = \sum_{i=1}^p \lambda_i(\hat{S})^{-1} = n\sum_{i=1}^p \sigma_i(X)^{-2}$$ where $\sigma_1(X), \dots, \sigma_p(X)$ are the nonzero singular values of $X$. Then we can use the inverse second moment identity (observed by Tao and Vu here http://arxiv.org/pdf/0807.4898v5.pdf; the rectangular case is given as Lemma 4.14 here: https://arxiv.org/pdf/1109.3343v4.pdf) to express $$\sum_{i=1}^p \sigma_i(X)^{-2}=\sum_{i=1}^p \text{dist}(R_i,R_{-i})^{-2}$$ where $\text{dist}(R_i,R_{-i})$ is the distance from the $i$th row of $X$ to the span of the remaining rows. Since $X$ has normally distributed entries the variables $\text{dist}(R_i,R_{-i})^{2}$ have chi-squared distribution, so the expectation of this sum can be computed explicitly. Unless I have made some mistake it comes out to $\mathbb{E}\frac1p\text{tr}(\hat{S}^{-1}) = \frac{n}{n-p-1} = \frac{1}{1-\alpha}+o(1)$ (note we need the normalization $1/p$).
The formula $$\mathbb E[(\sum_{i=1}^n x_ix_i^T)^{-1}] = I_p/(n-p-1)$$ is exact. It is known on Wikipedia as the expectation of the inverse wishart distribtion: https://en.wikipedia.org/wiki/Inverse-Wishart_distribution
To prove it, outside of the diagonal for some $$i\ne j$$: with $$X$$ the matrix with rows $$x_1,...,x_n$$ and $$e_j$$ the $$j$$-th canonical vector, the fact that $$X(I_p - 2e_je_j^T)$$ has the same distribution as $$X$$ (i.e., changing the sign of the $$j$$-th column) gives that the off-diagional $$i,j$$-th element is 0.
For the diagonal elements, by symmetry it is enough to compute $$trace[(X^TX)^{-1}]$$ as asked in the question. By Stein's formula $$\mathbb E[x_i^T F(X)]=\sum_j \mathbb E[ e_j^T \frac{\partial}{\partial x_{ij}} F(X)]$$ for $$F$$ valued in $$R^p$$ we find \begin{align} \mathbb E[x_i^T(X^TX)^{-1}x_i] =\mathbb E\Big[\sum_{j=1}^p e_j^T(X^TX)^{-1} e_j - e_j^T(X^TX)^{-1}(e_j x_i^T + x_i e_j^T)(X^TX)^{-1} x_i\Big]. \end{align} Summing over $$i=1,...,n$$ $$\mathbb E\sum_{i=1}^n x_i^T(X^TX)^{-1}x_i = n \mathbb E\Big[Tr[(X^TX)^{-1}] - Tr[(X^TX)^{-1}]\sum_{i=1}^n x_i^T(X^TX)^{-1}x_i - Tr[(X^TX)^{-1}\sum_{i=1}^nx_ix_i^T(X^TX)^{-1}] \Big]$$ or equivalently $$p=(n-p-1)\mathbb E Tr[(X^TX)^{-1}]$$ which is the desired identity.