5
$\begingroup$

There are many possible metrics one can place on the space of Gaussian probability measures on $\mathbb{R}^n$, with strictly positive definite co-variance matrices. Let's denote this space by $X$.

I'm particularly interested in the information geometric one (using the Fisher-Rao-Riemann metric) and the one induced by restricting the Wasserstein $2$ metric from $\mathcal{P}_2(\mathbb{R}^n)$ to the subspace $X$. But how doe these compare? Most specifically, is the Wasserstein $2$-distance dominated by the Fisher-Rao metric's induced distance function?

$\endgroup$

2 Answers 2

9
$\begingroup$

It is not the case that the Fisher-Rao distance dominates the Wasserstein distance. For instance, it fails for univariate normal distributions $\mathcal{N}(\mu,\sigma)$. In particular, the Wasserstein distance is the Euclidean distance on the half-plane $\mathbb{H}= \{(\mu,\sigma)~|~ \sigma >0\}$. On the other hand, the Fisher metric is hyperbolic, with metric $ds^2= \frac{1}{\sigma^2}(d \mu^2 + 2d \sigma^2).$ As a result, when the variance is large enough, the distance in the Fisher-Rao metric will be smaller than in the Wasserstein metric.

However, it is possible to bound the Wasserstein distance using the entropy, (or more precisely the KL-divergence). In particular, Talagrand proved that if $\gamma$ is the standard multivariate Gaussian with density $$d \gamma(x)=\frac{e^{-|x|^{2} / 2}}{(2 \pi)^{n / 2}} d x,$$ and $\mu$ is any measure with Radon-Nikodym derivative $\frac{d\mu}{d \gamma}= h$, then the following inequality holds

$$W(\mu, \gamma) \leq \sqrt{2 \int_{\mathbb{R}^{n}} h \log h \, d \gamma}.$$

For a more complete reference, Otto and Villani wrote an excellent paper generalizing this inequality and connecting it to the log-Sobolev inequality.

$\endgroup$
4
  • $\begingroup$ I should add that if you start with a different multivariate normal, the constant in the final inequality will depend on the covariance matrix, so the bound isn't uniform in the space of Gaussian measures (although it is uniform in the dimension). $\endgroup$
    – Gabe K
    Dec 28, 2020 at 16:28
  • $\begingroup$ Could you please provide a reference for the mentioned formulas? I mean the statements about metrics $\endgroup$ May 30 at 7:35
  • $\begingroup$ @ArtemAlexandrov I don't know of where the first result was originally proved, but the identity for the Gaussian measures can be found in "Wasserstein Geometry of Gaussian Measures" by Takatsu. The hyperbolic geometry of the Fisher metric in this setting is generally attributed to Amari. One reference for this is "Fisher information distance: a geometrical reading" by Costa, Santos and Strapasson. $\endgroup$
    – Gabe K
    Jun 29 at 18:16
  • $\begingroup$ Thanks! It is exactly what I want! $\endgroup$ Jun 29 at 20:11
2
$\begingroup$

Wasserstein distance has not good properties of invariances compared to Fisher Metric, that could be extended on convex cones by Jean-Louis Koszul tools (https://link.springer.com/chapter/10.1007/978-3-030-02520-5_12), and on homogeneous symplectic manifolds for Lie groups by Jean-Marie Souriau tools (https://www.mdpi.com/1099-4300/22/5/498). These topics are developd in GSI conferences: www.gsi2021.org

$\endgroup$
2
  • 1
    $\begingroup$ Yes true, but the Wasserstein distance has a nice dual representation which has "robust/adversarial" interpretations...this does not seem to be the case for the Fisher metric. $\endgroup$ Dec 28, 2020 at 15:42
  • 1
    $\begingroup$ In many situations the lack of invariance for the Wasserstein metric is a feature, not a bug. If you are comparing categorical datasets (e.g. draws from a multinomial distribution), then it makes sense to use the Fisher metric or something "entropic" which is invariant under sufficient statistics. However, in many applications the sample space has an underlying metric to take into account and if you choose a divergence with strong invariance properties, you will ignore the underlying geometry. $\endgroup$
    – Gabe K
    Dec 28, 2020 at 16:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.