MathOverflow will be down for maintenance for approximately 3 hours, starting Monday evening (06/24/2013) at approximately 9:00 PM Eastern time (UTC-4).
show/hide this revision's text 2 deleted 4 characters in body

De Finetti's theorem has already been mentioned, but it seems to me that it answers the original question. In this case, it says that any exchangeable infinite sequence $X_1$, $X_2$, $X_3$, ... X_1, X_2, X_3, \ldots$ of real-valued random variables comes from some probability measure $\Phi$ on the set of measures on $\Bbb R$. The sequence is generated by picking $\mu\sim\Phi$ and then taking i.i.d. $X_1$, $X_2$, $X_3$, . .. $\sim\mu$. X_1, X_2, X_3, \ldots \sim\mu$. So, the third bullet point is automatically satisfied, and the "population distribution" is $\mu$. To get marginal normality you only need that ${\bf E}[\mu]=N(0,1)$, so there is a wide choice for $\Phi$.

show/hide this revision's text 1

De Finetti's theorem has already been mentioned, but it seems to me that it answers the original question. In this case, it says that any exchangeable infinite sequence $X_1$, $X_2$, $X_3$, ... of real-valued random variables comes from some probability measure $\Phi$ on the set of measures on $\Bbb R$. The sequence is generated by picking $\mu\sim\Phi$ and then taking i.i.d. $X_1$, $X_2$, $X_3$, ... $\sim\mu$. So, the third bullet point is automatically satisfied, and the "population distribution" is $\mu$. To get marginal normality you only need that ${\bf E}[\mu]=N(0,1)$, so there is a wide choice for $\Phi$.