5
$\begingroup$

I am reading the survey of the relationships between metrics of distributions (see https://arxiv.org/pdf/math/0209021.pdf for the paper). The general results show that for general distributions, we cannot upper bound the total variation by Wasserstein distance. Many answers in MO give the same intuitive counterexample: consider a discrete distribution and a continuous distribution.

However, assuming the two underlying distributions are both continuous (for example consider the simpliest when they are both Gaussian mixtures with zeros mean or just Gaussians with zero mean), can we find such an upper bound?

Any related suggestions or comments are welcomed!

Improve this question
$\endgroup$

2 Answers 2

Reset to default
9
$\begingroup$

No. One should realize that the transportation and the total variation distances metrize two quite different topologies. Even if the measures are equivalent (i.e., absolutely continuous with respect to each other), one can still easily have examples when the transportation distance is arbitrarily close to 0, whereas the total variation distance is arbitrarily close to 2. Bounding the Radon-Nikodym derivatives is not of much help either as, once again, one can easily have examples of measures arbitrarily close in the transportation metric, whereas their total variation distance will be the one dictated by the density bounds. However, if you talk about measures from a certain specific class, then the situation may be different.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Is there a reference for a counter example example from the sort you mentioned? $\endgroup$
    – Amir Sagiv
    Dec 10, 2018 at 0:25
  • 2
    $\begingroup$ It's quite easy: the discrete counterexamples are pairs of $\delta$-measures $\delta_x$ and $\delta_y$ with close (but different) $x$ and $y$. In order to have a counterexample with equivalent measures take $\epsilon<d(x,y)/2$ and define measures $\lambda,\mu$ in such a way that $d\lambda/d\mu=C$ on the ball $B(x,\epsilon)$ and $d\lambda/d\mu=1/C$ on $B(y,\epsilon)$. $\endgroup$
    – R W
    Dec 10, 2018 at 1:23
4
$\begingroup$

If both probability measures have smooth densities, the total variation can be bounded by Wasserstein (even by a weaker metric Levy-Prokhorov), see the 2017 paper A novel approach to Bayesian consistency, by myself and Stephen G. Walker.

More precisely, see Lemma 5.1 and proof of Theorem 2.1. The basic idea is given below.

If both $p$ and $q$ are smooth densities, $d_V(p, p_\gamma)$ and $d_V(q, q_\gamma)$ are sufficiently small for every small gamma, where $d_V$ is the total variation and $p_\gamma$ is a convolution of p and the uniform density as defined in the paper.

Note that $d_V(p, q) < d_V(p, p_\gamma) + d_V(p_\gamma, q_\gamma) + d_v(q_\gamma, q)$ by the triangle inequality.

Lemma 5.1 guarantees that if $p$ and $q$ are close in Levy-Prokhorov metric, then $d_V(p_\gamma, q_\gamma)$ is also small provided that $\gamma$ is much bigger than the Levy-Prochorov distance.

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ Improved results are now published in doi.org/10.1016/j.spl.2020.108771. $\endgroup$
    – Minwoo Chae
    May 21, 2020 at 12:33
  • $\begingroup$ Do you also have multivariate versions of your results? $\endgroup$
    – ABIM
    Mar 1, 2021 at 12:57
  • 1
    $\begingroup$ Several researchers asked me about the multivariate extension, and now I have published another paper generalizing the previous results. Please see the following paper: doi.org/10.1016/j.spl.2024.110131 $\endgroup$
    – Minwoo Chae
    Apr 10 at 23:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.