I have been struggling quite a bit with reconciling my intuitive understanding of probability distributions with the weird properties that almost all topologies on probability distributions possess.

For example, consider a mixture random variable $X_n$: pick a Gaussian centered at 0 with variance 1, and with probability $\frac{1}{n}$, add $n$ to the result. A sequence of such random variables would converge (weakly and in total variation) to a Gaussian centered at 0 with variance 1, but the mean of the $X_n$ is always $1$ and the variances converge to $+\infty$. I really don't like saying that this sequence converges because of that.

edit: $X_n$ has density

$$p_n(x) = \frac{n-1}{n} g(x) + \frac{1}{n} g(x-n)$$

where $g$ is the density of the gaussian with unit variance and mean 0

I took me quite some time to remember everything I've forgotten about topologies, but I finally figured out what was so unsatisfying to me about such examples: the limit of the sequence is not a conventional distribution. In the example above, the limit is a weird "Gaussian of mean 1 and of infinite variance". In topological terms, the set of probability distributions isn't complete under the weak (and TV, and all the other topologies I've looked at).

(note:the problem remains with probability measures)

I then face the following question:

does there exist a topology such that the ensemble of probability distributions is complete ?

If no, does that absence reflect an interesting property of the ensemble of probability distributions ? Or is it just boring ?

**Original post here (crosspost crossvalidated!):** https://stats.stackexchange.com/questions/186670/topologies-for-which-the-ensemble-of-probability-distributions-is-complete

isreallyN(0,1). $\endgroup$ – Yoav Kallus Dec 20 '15 at 19:06completeness, it's an issue withcontinuity. I think you are really trying to ask: "Is there a topology on the space of probability distributions with the property that the functions $X \mapsto E(X)$ and $X \mapsto Var(X)$ defined in this space are continuous? $\endgroup$ – Paul Siegel Dec 20 '15 at 20:177more comments