I have been struggling quite a bit with reconciling my intuitive understanding of probability distributions with the weird properties that almost all topologies on probability distributions possess.

For example, consider a mixture random variable $X_n$: pick a Gaussian centered at 0 with variance 1, and with probability $\frac{1}{n}$, add $n$ to the result. A sequence of such random variables would converge (weakly and in total variation) to a Gaussian centered at 0 with variance 1, but the mean of the $X_n$ is always $1$ and the variances converge to $+\infty$. I really don't like saying that this sequence converges because of that.

edit: $X_n$ has density

$$p_n(x) = \frac{n-1}{n} g(x) + \frac{1}{n} g(x-n)$$

where $g$ is the density of the gaussian with unit variance and mean 0

I took me quite some time to remember everything I've forgotten about topologies, but I finally figured out what was so unsatisfying to me about such examples: the limit of the sequence is not a conventional distribution. In the example above, the limit is a weird "Gaussian of mean 1 and of infinite variance". In topological terms, the set of probability distributions isn't complete under the weak (and TV, and all the other topologies I've looked at).

(note:the problem remains with probability measures)

I then face the following question:

• does there exist a topology such that the ensemble of probability distributions is complete ?

• If no, does that absence reflect an interesting property of the ensemble of probability distributions ? Or is it just boring ?

Original post here (crosspost crossvalidated!): https://stats.stackexchange.com/questions/186670/topologies-for-which-the-ensemble-of-probability-distributions-is-complete