Skip to main content
2 of 4
added 738 characters in body
Iosif Pinelis
  • 127.8k
  • 8
  • 107
  • 229

$\newcommand{\tr}{\operatorname{tr}}$ Let $x=[x^1,\dots,x^p]^T:=x_1$, $y:=xx^T$, $\mu:=Ey$, $w:=y-\mu$, and $s:=\sum_1^n x_ix_i^T$. Then, by the appropriate laws of large numbers, $s/n\to Ey$ almost surely and hence in probability and in distribution, provided that $Ey$ exists in $\mathbb R^{p\times p}$. Everywhere here, the convergence is for $n\to\infty$.

Assume now that $E|x|^4<\infty$, where $|x|$ is the Euclidean norm of $x$. Note that $y$ is a random matrix in the vector space $\mathbb R^{p\times p}$, which is naturally endowed by the inner product $A\cdot B:=\tr AB^T$ for $A$ and $B$ in $\mathbb R^{p\times p}$. With respect to this inner product, the covariance operator (say $R$) of the random matrix $y$ is given by the formula $$(Rf)_{k,l}=\sum_{i,j=1}^pf_{i,j}\,Cov(x^ix^j,x^kx^l) $$ for $f=(f_{i,j})\in\mathbb R^{p\times p}$ and $k,l=1,\dots,p$. So, by the multivariate central limit theorem, $(s-n\mu)/\sqrt n$ converges in distribution to a Gaussian random vector in $\mathbb R^{p\times p}$ with covariance operator $R$.

Iosif Pinelis
  • 127.8k
  • 8
  • 107
  • 229