Upper bound total variation by Wasserstein distance for continuous distance

Question

I am reading the survey of the relationships between metrics of distributions (see https://arxiv.org/pdf/math/0209021.pdf for the paper). The general results show that for general distributions, we cannot upper bound the total variation by Wasserstein distance. Many answers in MO give the same intuitive counterexample: consider a discrete distribution and a continuous distribution.

However, assuming the two underlying distributions are both continuous (for example consider the simpliest when they are both Gaussian mixtures with zeros mean or just Gaussians with zero mean), can we find such an upper bound?

Any related suggestions or comments are welcomed!

Answer 1

No. One should realize that the transportation and the total variation distances metrize two quite different topologies. Even if the measures are equivalent (i.e., absolutely continuous with respect to each other), one can still easily have examples when the transportation distance is arbitrarily close to 0, whereas the total variation distance is arbitrarily close to 2. Bounding the Radon-Nikodym derivatives is not of much help either as, once again, one can easily have examples of measures arbitrarily close in the transportation metric, whereas their total variation distance will be the one dictated by the density bounds. However, if you talk about measures from a certain specific class, then the situation may be different.

Answer 2

If both probability measures have smooth densities, the total variation can be bounded by Wasserstein (even by a weaker metric Levy-Prokhorov), see the 2017 paper A novel approach to Bayesian consistency, by myself and Stephen G. Walker.

More precisely, see Lemma 5.1 and proof of Theorem 2.1. The basic idea is given below.

If both $p$ and $q$ are smooth densities, $d_V(p, p_\gamma)$ and $d_V(q, q_\gamma)$ are sufficiently small for every small gamma, where $d_V$ is the total variation and $p_\gamma$ is a convolution of p and the uniform density as defined in the paper.

Note that $d_V(p, q) < d_V(p, p_\gamma) + d_V(p_\gamma, q_\gamma) + d_v(q_\gamma, q)$ by the triangle inequality.

Lemma 5.1 guarantees that if $p$ and $q$ are close in Levy-Prokhorov metric, then $d_V(p_\gamma, q_\gamma)$ is also small provided that $\gamma$ is much bigger than the Levy-Prochorov distance.

Answer 3

To complement the existing answers, the following example shows that a bound $$ \|\mu - \nu\|_{1} ≤ C W_2(\mu, \nu) \tag{1} $$ does not hold even when we restrict the densities to smooth, globally Lipschitz continuous functions with a Lipschitz constant less than a fixed value. Indeed, consider the setting where $$ \mu_L = \frac{1}{L} \mathbf 1_{[0, L]}(x) \Bigl( 1 - \cos(2\pi L x) \Bigr), \qquad \nu_L = \mu_L \Bigl( x - \frac{1}{2L} \Bigr) $$ for $L$ a natural number. For all values of $L$, these functions are Lipschitz continuous with constant $2\pi$. In this example, $d_1(\mu_L, \nu_L)$ is constant with $L$, but the Wasserstein distance $W_2(\mu_L, \nu_L)$ tends to 0 as $L \to \infty$. See the illustration below.

Even if we restrict our attention to the one-dimensional torus, the bound (1) does not hold. Indeed consider the following other example: $$ \mu_L = 1 + \frac{\cos(2\pi L x)}{L}, \qquad \nu_L = \mu_L \Bigl( x - \frac{1}{2L} \Bigr) $$ Then $d_1(\mu_L, \nu_L)$ scales as $1/L$ as $L \to \infty$, but $W_2(\mu_L, \nu_L)$ scales as $1/L^2$. Illustration below: