# How does a statistical divergence change under a Lipschitz push-forward map?

Let $$\mu, \nu$$ be two probability measures on a space $$X$$ (assume Polish space). $$T: X \rightarrow Y$$ is a Lipschitz-map that acts as a push-forward on these measures; let $$\mu^\prime = T_{\#\mu}$$ and $$\nu^\prime = T_{\#\nu}$$ be the resulting push-forward measures on the space $$Y$$.

Let $$d(\mu, \nu)$$ denote some distance function between measures. What can be said about how $$d(\mu, \nu)$$ and $$d(\mu^\prime, \nu^\prime)$$ relate to each other? Especially, how is the relationship a function of the Lipschitz constant of $$T$$?

Of course the relationship is likely to be a function of the exact nature of $$d()$$ (e.g., KL-divergence, or Total-variation or Wassterstein .. ). I suspect a lot must be known about this type of questions; what would be the right place to look at?

## 1 Answer

If $$d(\mu,\nu)$$ is taken to be the total variation metric, then Lipschitz and metric properties don't matter. This is due to a "data processing inequality" of sorts: applying a transformation can only make two distributions closer in TV. I'll illustrate this for discrete sets $$X,Y$$: $$d(\mu',\nu') =\sum_{y\in Y}|\mu'(y)-\nu'(y)| =\sum_{y\in Y}|\sum_{x\in T^{-1}(y)}\mu(x)-\nu(x)| \le\sum_{x\in X}|\mu(x)-\nu(x)|=d(\mu,\nu).$$

For other metrics, the Lipschitz constant will matter. For example, take two densities $$\mu,\nu$$ on the real line and put $$T(x)=a x$$ for some $$a>0$$. Then $$d_p(\mu',\nu'):=(\int |\mu'-\nu'|^p)^{1/p} = a^{(1-p)/p}d_p(\mu,\nu).$$ The last example is taken from Ch. 1 of https://www.szit.bme.hu/~gyorfi/nonpar_dens_en.html ; you may find the discussion there useful.