Topologies for which the ensemble of probability measures is complete I have been struggling quite a bit with reconciling my intuitive understanding of probability distributions with the weird properties that almost all topologies on probability distributions possess.
For example, consider a mixture random variable $X_n$: pick a Gaussian centered at 0 with variance 1, and with probability $\frac{1}{n}$, add $n$ to the result. A sequence of such random variables would converge (weakly and in total variation) to a Gaussian centered at 0 with variance 1, but the mean of the $X_n$ is always $1$ and the variances converge to $+\infty$. I really don't like saying that this sequence converges because of that.
edit: $X_n$ has density
$$p_n(x) = \frac{n-1}{n} g(x) + \frac{1}{n} g(x-n)$$
where $g$ is the density of the gaussian with unit variance and mean 0
I took me quite some time to remember everything I've forgotten about topologies, but I finally figured out what was so unsatisfying to me about such  examples: the limit of the sequence is not a conventional distribution. In the example above, the limit is a weird "Gaussian of mean 1 and of infinite variance". In topological terms, the set of probability distributions isn't complete under the weak (and TV, and all the other topologies I've looked at).
(note:the problem remains with probability measures)
I then face the following question:


*

*does there exist a topology such that the ensemble of probability distributions is complete ?

*If no, does that absence reflect an interesting property of the ensemble of probability distributions ? Or is it just boring ?
Original post here (crosspost crossvalidated!): https://stats.stackexchange.com/questions/186670/topologies-for-which-the-ensemble-of-probability-distributions-is-complete
 A: As I asserted in my comments, I think it is too much to hope for a reasonable topology on the space of random variables which makes the map $X \mapsto E(X)$ continuous.  This is a bit like hoping that pointwise convergence or convergence in measure implies convergence in the $L^1$ norm; it seems reasonable, but there are simple counter-examples.
But all is not lost.  In analysis one salvages the situation by making stronger assumptions: for instance, if $f_n \to f$ pointwise and $|f_n(x)| \leq g(x)$ for some integrable function $g$ and all $n$, then $\int f_n \to \int f$ (the dominated convergence theorem).  There is a sort of counterpart to this in probability theory.
Definition: A sequence $X_n$ of random variables is uniformly integrable if:


*

*$E(|X_n|)$ is uniformly bounded in $n$: there is a constant $K$ such that $E(|X_n|) \leq K$ for all $n$

*For every $\varepsilon > 0$ there exists $\delta > 0$ such that $\int_A |X_n| < \varepsilon$ for all $n$ whenever $P(A) < \delta$


Uniform integrability is implied by the stronger (but more easily checked) condition that $E(|X_n|^{1 + \delta})$ is uniformly bounded for some $\delta > 0$.
Theorem: Suppose $X_n$ converges to $X$ in distribution and the sequence $|X_n|^k$ is uniformly integrable.  Then $E(X_n^j) \to E(X^j)$ whenever $1 \leq j \leq k$.  (Reference)
There are some other results of this flavor in the wikipedia page on convergence of random variables but this theorem is the best result that I know.
