Does the average of correlated Gaussian random variables with mean zero and different variances converge in probability to their mean?

Question

Let $X_i\sim N(0,\sigma_i^2)$ and $\operatorname{Corr}(X_i,X_j)>0$. Is it possible to show that $$\frac{1}{N} \sum_{i=1}^N X_i \overset{p}\rightarrow E[X_i]=0.$$ Do you have a reference to a law of large numbers that can be applied in this case?

Answer 1

The following reference has several versions of the WLLN:

Serfling, Robert J. 1980. Approximation Theorems of Mathematical Statistics. New York: Wiley.

It refers to the following for a version with correlated $X_i$:

Serfling, R. J. 1970. “Convergence Properties of Sn Under Moment Restrictions.” The Annals of Mathematical Statistics 41 (4): 1235–48.

It goes like this: Let $X_1, X_2, \dots$ have means $\mu_1, \mu_2, \dots$, variances $\sigma_1^2, \sigma_2^2, \dots$, and covariances $\text{Cov}(X_i,X_j)$ satisfying $$ \text{Cov}(X_i,X_j) \leq \rho_{j-i}\sigma_i\sigma_j $$ for $i \leq j$, where $0 \leq \rho_k \leq 1$ for all $k = 0, 1, \dots$. If $\sum_{i=1}^\infty \rho_i$ and $\sum_{i=1}^\infty \sigma_i^2(\log i)^2/i^2$ are both convergent, then $$ \frac1n \sum_{i=1}^n X_i - \frac1n \sum_{i=1}^n \mu_i \to 0 $$ with probability 1 (and therefore in probability as well).

For a sense of why this cannot hold without conditions on the covariances, imagine $\mathbb{E}[X_i] = \mu$ for all $i$ and $\text{Var}(X_i) = \sigma^2$ so that there is a common mean and variance, but $\text{Cov}(X_i,X_j) = \sigma^2$ for all $i,j$. Then the $X_i$ will all be perfectly correlated and therefore identical, and you'll have $\frac1n \sum_{i=1}^n X_i = X_1$ for all $n$, but this is not the common mean $\mu$ (or any fixed number at all).

Answer 2

Let $\sigma_{i,j} = \operatorname{cov}(X_i,X_j) $ for $i,j\in\{\,1,2,3,\ldots\,\}.$ (In particular $\sigma_{i,i}= \operatorname{var}(X_i).$

Let $\overline X_n= \dfrac{X_1+\cdots+X_n} n.$

Then $$ \operatorname{var} \overline X_n = \frac 1 {n^2} \left( \sum_{(i,j) \, \in \, \{1,\,\ldots\,,\,n\}^2} \sigma_{i,j} \right). \tag1 $$ If it were assumed that all covariances are zero and all variances are equal, then this would plainly converge to $0$ as $n\to\infty.$ In that case an application of Chebyshev's inequality will show that $\overline X_n$ converges in probability to $0$ as $n\to\infty.$

The question reduces to this: For which arrays $\big(\sigma_{ij}\big)_{(i,j)\in\mathbb N^2}$ whose every upper-left corner is positive-semidefinite does line $(1)$ above converge to $0$ as $n\to\infty\text{?}$