I am reading the survey of the relationships between metrics of distributions (see https://arxiv.org/pdf/math/0209021.pdf for the paper). The general results show that for general distributions, we cannot upper bound the total variation by Wasserstein distance. Many answers in MO give the same intuitive counterexample: consider a discrete distribution and a continuous distribution.

However, assuming the two underlying distributions are both continuous (for example consider the simpliest when they are both Gaussian mixtures with zeros mean or just Gaussians with zero mean), can we find such an upper bound?

Any related suggestions or comments are welcomed!


No. One should realize that the transportation and the total variation distances metrize two quite different topologies. Even if the measures are equivalent (i.e., absolutely continuous with respect to each other), one can still easily have examples when the transportation distance is arbitrarily close to 0, whereas the total variation distance is arbitrarily close to 2. Bounding the Radon-Nikodym derivatives is not of much help either as, once again, one can easily have examples of measures arbitrarily close in the transportation metric, whereas their total variation distance will be the one dictated by the density bounds. However, if you talk about measures from a certain specific class, then the situation may be different.

  • $\begingroup$ Is there a reference for a counter example example from the sort you mentioned? $\endgroup$ – Amir Sagiv Dec 10 '18 at 0:25
  • 1
    $\begingroup$ It's quite easy: the discrete counterexamples are pairs of $\delta$-measures $\delta_x$ and $\delta_y$ with close (but different) $x$ and $y$. In order to have a counterexample with equivalent measures take $\epsilon<d(x,y)/2$ and define measures $\lambda,\mu$ in such a way that $d\lambda/d\mu=C$ on the ball $B(x,\epsilon)$ and $d\lambda/d\mu=1/C$ on $B(y,\epsilon)$. $\endgroup$ – R W Dec 10 '18 at 1:23

If both probability measures have smooth densities, the total variation can be bounded by Wasserstein (even by a weaker metric Levy-Prokhorov), see this paper.

More precisely, see Lemma 5.1 and proof of Theorem 2.1. The basic idea is given below.

If both $p$ and $q$ are smooth densities, $d_V(p, p_\gamma)$ and $d_V(q, q_\gamma)$ are sufficiently small for every small gamma, where $d_V$ is the total variation and $p_\gamma$ is a convolution of p and the uniform density as defined in the paper.

Note that $d_V(p, q) < d_V(p, p_\gamma) + d_V(p_\gamma, q_\gamma) + d_v(q_\gamma, q)$ by the triangle inequality.

Lemma 5.1 guarantees that if $p$ and $q$ are close in Levy-Prokhorov metric, then $d_V(p_\gamma, q_\gamma)$ is also small provided that $\gamma$ is much bigger than the Levy-Prochorov distance.

  • $\begingroup$ Hi, welcome to MO. Where in your paper is the bound you mention in the answer? $\endgroup$ – Amir Sagiv Jan 31 '19 at 18:10
  • 1
    $\begingroup$ Improved results are now published in doi.org/10.1016/j.spl.2020.108771. $\endgroup$ – Minwoo Chae May 21 '20 at 12:33
  • $\begingroup$ Do you also have multivariate versions of your results? $\endgroup$ – BLBA Mar 1 at 12:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.