exchangeable normal r.v.s

Question

Usually "exchangeable normal random variables" means jointly normal random variables $X_1,\ldots,X_n$ (i.e. so distributed that every linear combination of them is normally distributed) that are exchangeable in the sense that if no matter you permute the indices, you don't alter the probability distribution of the $n$-tuple.

But I wonder about a different sort of exchangeable normals. Is there some probability distribution of an infinite sequence $X_0,X_1,X_2,\ldots$ of random variables such that

$\bullet$ For each $i$, $X_i \sim N(0,1)$ (marginal normality),

$\bullet$ Finite permutations of the indices never alter the probability distribution of the sequence as a whole (exchangeability),

$\bullet$ For any Borel set $A$, $\lim\limits_{n\to\infty}\Pr(X_0\in A \mid X_1,\ldots,X_n) = \lim\limits_{n\to\infty} \dfrac{|A\cap \lbrace X_1,\ldots,X_n \rbrace|}{n}$

$\bullet$ Any reasonably well behaved distribution could be the limiting distribution; which one it is would determine the nature of the dependence among $X_1,X_2,X_3,\ldots.$

?

(If I'm not mistaken, it would be enough to show the third bullet point is satisfied whenever $A$ is a half-infinite interval.)

Later edit: At least initially, I'm leaning toward construing "reasonably well behaved" as meaning having an everywhere strictly positive density with respect to Lebesgue measure (or---what is the same thing---a strictly positive density with respect to the marginal distribution). And I wonder if the answer might changed if we required it to have the same mean and variance as the marginal distribution. I'm thinking of the limiting distribution as a "population distribution", about whose nature one becomes less uncertain as the sample size grows.

Still later edit: I'm leaning toward construing the fourth bullet point something like this: You pick any not too nasty absolutely continuous probability distribution. Pick an infinite i.i.d. sample from it. Feed that sample into what you see above, i.e. find $\lim\limits_{n\to\infty} \Pr(X_0\in A\mid X_1=x_1, X_2 = x_2, \ldots, X_n=x_n)$ where $x_1,x_2,x_3,\ldots$ is what you got. Then the third bullet point should be satisfied. Then question is: can we guarantee that by some judicious choice of the joint probability distribution right at the outset, also satisfying the other bullet points?

Answer 1

De Finetti's theorem has already been mentioned, but it seems to me that it answers the original question. In this case, it says that any exchangeable infinite sequence $X_1, X_2, X_3, \ldots$ of real-valued random variables comes from some probability measure $\Phi$ on the set of measures on $\Bbb R$. The sequence is generated by picking $\mu\sim\Phi$ and then taking i.i.d. $X_1, X_2, X_3, \ldots \sim\mu$. So, the third bullet point is automatically satisfied, and the "population distribution" is $\mu$. To get marginal normality you only need that ${\bf E}[\mu]=N(0,1)$, so there is a wide choice for $\Phi$.

Answer 2

I will expand on this answer later if there is interest and when I have some references handy. But for now you may be interested in the following way of thinking about the problem.

One way to characterize the Gaussian distribution is as the unique distribution on $\mathbb{R}$ satisfying spherical symmetry. More precisely, for $N$ observations, consider the two-dimensional statistic $$T(X_{1:N}) = \left(\sum_{i=1}^N X_i, \sum_{i=1}^N X_j^2 \right).$$ Assume that the conditional distribution of the vector $X_{1:N}$ is uniform on the hypersphere with center $(T_1, \dots, T_1)$ and radius $\sqrt{(T_2 - T_1^2/N)}$. This implies that the density for $X_{1:N}$ may be written as $$f(X_{1:N}) = \int \prod_{j=1}^N \left[(2\pi)^{-\frac{1}{2}}\sigma^{-1} \exp{\left\lbrace\frac{(x_j - \mu)^2}{\sigma^2}\right\rbrace}\right] dP(\sigma, \mu)$$ for some density $dP(\sigma,\mu)$. (This sort of representation theorem motivates the use of prior distributions in Bayesian statistics.) Note that as $N \rightarrow \infty$, $T$ converges to the true first and second moments, which gives back another common characterization of the normal distribution as being specifiable using only them.

So, when you say that "usually 'exchangeable normal random variables' means jointly normal random variables" it makes me wonder what is the more critical property, the permutation invariance -- which does not uniquely define the distribution -- or the underlying symmetry -- which in the case of the normal distribution does. The reason I brought up the copula earlier is that I think getting your necessary marginals to be whatever is not much of a barrier, because you can always transform things elementwise. This makes me think that you are really asking whether or not there are other forms of exchangeable distributions of real-valued vectors, and there definitely are. Following the example of the spherical symmetry, the basic recipe is to specify a statistic and then specify a uniform transition kernel given the value of that statistic. This approach has been systematized by Steffen Lauritzen in a monograph S. L. Lauritzen. Extremal Families and Systems of Sufficient Statistics. Lecture Notes in Statistics, No. 49. A good textbook treatment of this is given in section 2.4 of Mark Schervish's Theory of Statistics (available on Google Books, but my toolbar for providing links seems to have vanished).

Apologies if you knew all of the above and I missed the point of your question, but your comment to Yuri makes me think that this stuff would be of interest. The keywords you'd want to include to dig around more include "de Finetti theorems", "extremal families", and "partial exchangeability".

Answer 3

i.i.d. standard normals seem to work, don't they?