Limiting distribution of "scatter matrix" $\frac{1}{n}XX^T:=\frac{1}{n}\sum_{i=1}^nx_ix_i^T$ for iid $x_1,\ldots,x_n \in \mathbb R^p$ Let $x_1,\ldots,x_n$ be drawn iid from such "nice" distribution on $\mathbb R^p$ (but possibly very general!), and let $X$ be the $n$-by-$p$ matrix formed by vertically stacking the $x_i$'s.

Question. What is the limiting distribution of the "scatter matrix" $\frac{1}{n}XX^T:=\frac{1}{n}\sum_{i=1}^nx_ix_i^T$ as $n \rightarrow \infty$ ?

Observations


*

*If the $x_i$'s are from a centered multivariate Gaussian, then $XX^T$ has a Wishart distribution.

 A: $\newcommand{\tr}{\operatorname{tr}}$
Let $x=[x^1,\dots,x^p]^T:=x_1$, $y:=xx^T$, $\mu:=Ey$, $w:=y-\mu$, and $s:=\sum_1^n x_ix_i^T$.
Then, by the appropriate laws of large numbers, $s/n\to Ey$ almost surely and hence in probability and in distribution, provided that $Ey$ exists in $\mathbb R^{p\times p}$. Everywhere here, the convergence is for $n\to\infty$.
Assume now that $E|x|^4<\infty$, where $|x|$ is the Euclidean norm of $x$. Note that $y$ is a random matrix in the vector space $\mathbb R^{p\times p}$, which is naturally endowed by the inner product $A\cdot B:=\tr AB^T$ for $A$ and $B$ in $\mathbb R^{p\times p}$. With respect to this inner product, the covariance operator (say $R$) of the random matrix $y$ is given by the formula
$$(Rf)_{k,l}=\sum_{i,j=1}^pf_{i,j}\operatorname{Cov}(x^ix^j,x^kx^l)
$$
for $f=(f_{i,j})\in\mathbb R^{p\times p}$ and $k,l=1,\dots,p$. So, by the multivariate central limit theorem, $(s-n\mu)/\sqrt n$ converges in distribution to a zero-mean Gaussian random matrix in $\mathbb R^{p\times p}$ with covariance operator $R$.
