# Distance between distributions and distance of moments

Let's say I have a sequence of random variables $X_n$ such that $$\mathbf E X_n^k = \mathbf E X^k+O(a_k/\sqrt{n})\quad\text{for all }k\in\mathbb N,\tag{\ast}$$ where $X$ is a random variable of standard (zero mean, unit variance) Gaussian distribution and $a_k$ are some constants which typically grow with $k$ (for my purposes they would be $a_k=(k/2)!$, for example).

Since the normal distribution is uniquely determined by its moments $(\ast)$ implies that $X_n\Rightarrow X$ weakly, as $n\to\infty$.

Is there an appropriate way to formalize the distance $d$ between the distributions of $X_n$ and $X$, such that $(\ast)$ implies $$d(X_n,X)=O(f(n))$$ for some function $f$? I think $(\ast)$ is not strong enough to control distance measures like the Kullback-Leibler divergence or the Hellinger distance, but there might be some appropriate weaker notion?

I think that $(\ast)$ implies $$\mathbf E f(X_n)=\mathbf E f(X)+O(1/\sqrt{n})$$ for a small class of test functions $f$, but depending on the growth of $a_k$, this class might be vary small.

• How about the sup of modulus of the difference of characteristic functions over some complex disc around the origin? It's not a metric on the whole space of probability measures but on a space that seems big enough for your purposes. – Abdelmalek Abdesselam Nov 8 '16 at 23:43

## 2 Answers

The natural thing to compare in this context seems to be the moment generating functions of $X_n$ and $X$. In particular, consider: \begin{align*} \mathbf{E} \exp(t X_n) - \mathbf{E} \exp(t X) &= \sum_{k=0}^{\infty} \frac{t^k}{k!} \left( \mathbf{E} X_n^k - \mathbf{E} X^k \right) \\ &\le \frac{1}{\sqrt{n}} \sum_{k=0}^{\infty} \frac{t^k}{k!} (k/2)! = \frac{1}{\sqrt{n}} \left( 1 + e^{t^2/4} \sqrt{\pi} t + e^{t^2/4} \sqrt{\pi} t \operatorname{Erf}(t/2) \right) \end{align*} where we used the hypothesis given by the OP.

Given this bound on their moment generating functions (or a similar bound on the characteristic function), to what extent do the laws of $X_n$ and $X$ agree? There seems to be an interesting discussion about this in the statistics literature.

• McCullagh, Peter. "Does the moment-generating function characterize a distribution?" The American Statistician 48.3 (1994): 208-208.
• Waller, Lance A. "Does the characteristic function numerically distinguish distributions?" The American Statistician 49.2 (1995): 150-152.
• Luceño, Alberto. "Further evidence supporting the numerical usefulness of characteristic functions." The American Statistician 51.3 (1997): 233-234.
• I was not aware about this discussion in the statistics literature. Especially the McCullagh paper is somewhat surprising. Thanks for the interesting references! – Julian Nov 9 '16 at 9:12
• A further comment: the above bound implies that the Lévy distance between $X_n$ and $X$ can be bounded by $c\frac{\log\log n}{\sqrt{\log n}}$ for some constant $c$. – Julian Nov 10 '16 at 13:46
• @Julian A characterization of the Lévy distance I am familiar with is the one given in Problem 26.15 of the 4th edition of Probability and Measure by P. Billingsley. However, I don't see how to combine the above bound and this characterization to obtain this result. Could you please clarify? – Nawaf Bou-Rabee Nov 10 '16 at 14:55
• This follows from the inequality $d_L(X_n,X)\leq \frac{1}{\pi}\int_{0}^T \lvert \frac{\phi_n(t)-\phi(t)}{t}\rvert dt + \frac{2e\log T}{T}$ which holds for $T\geq 2$ and where $\phi_n, \phi$ are the characteristic functions of $X_n,X$. The above form is from Theorem 1.4.13., N. G. Ushakov. Selected topics in characteristic functions. Walter de Gruyter, 2011 – Julian Nov 10 '16 at 22:47

The classical approach to this is called "smoothing" and is described in Chapter 16 of Feller's Introduction to Probability Theory, Vol 2. Basically you use the bounds on the moment differences to bound the difference between the characteristic functions near the origin, then adjust a parameter (called $T$ by Feller) to get the best result.