# Average Multivariate Gaussian

Suppose we have a (possibly infinite) collection k-variate gaussian distributions $\{(\mathcal{N}(\mu_{\lambda}, \Sigma_{\lambda}))\}$ ($\lambda$ is just a label), and for each distribution $\mu \in [-1,1]^k$, the variance of each coordinate $X_i$ is $1$, and all other covariances $Cov[X_i ,X_j] \in [-1,1]$.

Suppose also we have a distribution $\Lambda$ over labels $\lambda$. Sample a vector $X \in \mathbb{R}^d$ by first sampling $\lambda \sim \Lambda$, and then sampling $X \sim \mathcal{N}(\mu_{\lambda}, \Sigma_{\lambda})$.

Now, sample $Y$ from $\mathcal{N}(\mathbb{E}_{\lambda \sim \Lambda}[(\mu_\lambda, \Sigma_\lambda)])$.

Are $X$ and $Y$ ever identically distributed?

• I think the answer is "no" in nontrivial cases because the variance of the mixture is greater than the variances of the things being mixed on account of the variance of $\mu_\lambda$. But this is not a proof. – Brendan McKay Aug 6 '15 at 0:00

No, a mixture of Gaussians is not Gaussian: $\exp[-(x-1)^2]+\exp[-(x+1)^2]$ is very different from a constant times $\exp[-x^2]$.
• You're right, it's my bad understanding of "ever". I was about to give the same example. But with the conditions given here ($\mu\in[-1,1]$, unit variance) I bet the answer is no for $k=d=1$. – Jean Duchon Aug 5 '15 at 14:32
A sum of independent random variables is Gaussian if and only if each of them is Gaussian (Cramér, 1939). Apply this to $X_1$, which is the sum of a centered Gaussian with variance $1$ and an independent random mean $\mu_1$: $X_1$ would be Gaussian only if $\mu_1$ was...
Except of course in the trivial case of a non random label $\lambda=\lambda_0\ a.s.$
In view of Brendan's comment to the question, in nontrivial cases the answer is no for two reasons: $X_i$'s variance is greater than $Y_i$'s (even if both were Gaussian, which could happen if the condition $\mu_{\lambda,i}\in[-1,1]$ was suppressed), and $X_i$ cannot be Gaussian (of any variance) if $\mu_{\lambda,i}$ is not Gaussian.