Usually "exchangeable normal random variables" means *jointly* normal random variables $X_1,\ldots,X_n$ (i.e. so distributed that every linear combination of them is normally distributed) that are exchangeable in the sense that if no matter you permute the indices, you don't alter the probability distribution of the $n$-tuple.

But I wonder about a different sort of exchangeable normals. Is there some probability distribution of an infinite sequence $X_0,X_1,X_2,\ldots$ of random variables such that

$\bullet$ For each $i$, $X_i \sim N(0,1)$ (marginal normality),

$\bullet$ Finite permutations of the indices never alter the probability distribution of the sequence as a whole (exchangeability),

$\bullet$ For any Borel set $A$, $\lim\limits_{n\to\infty}\Pr(X_0\in A \mid X_1,\ldots,X_n) = \lim\limits_{n\to\infty} \dfrac{|A\cap \lbrace X_1,\ldots,X_n \rbrace|}{n}$

$\bullet$ Any reasonably well behaved distribution could be the limiting distribution; which one it is would determine the nature of the dependence among $X_1,X_2,X_3,\ldots.$

?

(If I'm not mistaken, it would be enough to show the third bullet point is satisfied whenever $A$ is a half-infinite interval.)

**Later edit:** At least initially, I'm leaning toward construing "reasonably well behaved" as meaning having an everywhere strictly positive density with respect to Lebesgue measure (or---what is the same thing---a strictly positive density with respect to the marginal distribution). And I wonder if the answer might changed if we required it to have the same mean and variance as the marginal distribution. I'm thinking of the limiting distribution as a "population distribution", about whose nature one becomes less uncertain as the sample size grows.

**Still later edit:** I'm leaning toward construing the fourth bullet point something like this: You pick any not too nasty absolutely continuous probability distribution. Pick an infinite i.i.d. sample from it. Feed that sample into what you see above, i.e. find $\lim\limits_{n\to\infty} \Pr(X_0\in A\mid X_1=x_1, X_2 = x_2, \ldots, X_n=x_n)$ where $x_1,x_2,x_3,\ldots$ is what you got. Then the *third* bullet point should be satisfied. Then question is: can we guarantee that by some judicious choice of the joint probability distribution right at the outset, also satisfying the other bullet points?

beforeinterpreting the TeX, so that \\ sends \ to the TeX interpreter, and \\{ sends \{ as desired. But I don't know why. $\endgroup$ – David M Kaplan Sep 7 '11 at 19:461more comment