Skip to main content
added 8 characters in body
Source Link
Michael Hardy
  • 1
  • 12
  • 85
  • 126

With regard to your first question, one such result shows that under some conditions on $q$, the following inequality holds for all p:

$W_2(p,q) \leq \sqrt{\frac{ KL(p||q)}{\rho}}$$$W_2(p,q) \leq \sqrt{\frac{ KL(p\parallel q)}\rho}$$

This is a so called Talagrand$(\rho)$ inequality, which holds whenever a suitable log-Sobolev inequality holds for $q$. For more information, the paper of Otto and Villani [1] is a good reference. This is all contained in Villani's book on optimal transport as well. This also contains many other inequalities between the various distances and divergences, so it is a good reference. However, due to the work of Otto, the Wasserstein 2 metric induces a formal Riemannian metric on the space of densities, and its geometry can be quite different from that of the Fisher-Rao metric.

As to your second question, the Fisher metric is unique in that it is the quadratic term of the Taylor series for any $f$-divergence. For more information on this phenomena, see [2].

[1] Otto, F., & Villani, C. (2000). Generalization of an inequality by Talagrand and links with the logarithmic Sobolev inequality. Journal of Functional Analysis, 173(2), 361-400.

[2] The Fisher Metric Will Not Be Deformed https://golem.ph.utexas.edu/category/2018/05/the_fisher_metric_will_not_be.html

With regard to your first question, one such result shows that under some conditions on $q$, the following inequality holds for all p:

$W_2(p,q) \leq \sqrt{\frac{ KL(p||q)}{\rho}}$

This is a so called Talagrand$(\rho)$ inequality, which holds whenever a suitable log-Sobolev inequality holds for $q$. For more information, the paper of Otto and Villani [1] is a good reference. This is all contained in Villani's book on optimal transport as well. This also contains many other inequalities between the various distances and divergences, so it is a good reference. However, due to the work of Otto, the Wasserstein 2 metric induces a formal Riemannian metric on the space of densities, and its geometry can be quite different from that of the Fisher-Rao metric.

As to your second question, the Fisher metric is unique in that it is the quadratic term of the Taylor series for any $f$-divergence. For more information on this phenomena, see [2].

[1] Otto, F., & Villani, C. (2000). Generalization of an inequality by Talagrand and links with the logarithmic Sobolev inequality. Journal of Functional Analysis, 173(2), 361-400.

[2] The Fisher Metric Will Not Be Deformed https://golem.ph.utexas.edu/category/2018/05/the_fisher_metric_will_not_be.html

With regard to your first question, one such result shows that under some conditions on $q$, the following inequality holds for all p:

$$W_2(p,q) \leq \sqrt{\frac{ KL(p\parallel q)}\rho}$$

This is a so called Talagrand$(\rho)$ inequality, which holds whenever a suitable log-Sobolev inequality holds for $q$. For more information, the paper of Otto and Villani [1] is a good reference. This is all contained in Villani's book on optimal transport as well. This also contains many other inequalities between the various distances and divergences, so it is a good reference. However, due to the work of Otto, the Wasserstein 2 metric induces a formal Riemannian metric on the space of densities, and its geometry can be quite different from that of the Fisher-Rao metric.

As to your second question, the Fisher metric is unique in that it is the quadratic term of the Taylor series for any $f$-divergence. For more information on this phenomena, see [2].

[1] Otto, F., & Villani, C. (2000). Generalization of an inequality by Talagrand and links with the logarithmic Sobolev inequality. Journal of Functional Analysis, 173(2), 361-400.

[2] The Fisher Metric Will Not Be Deformed https://golem.ph.utexas.edu/category/2018/05/the_fisher_metric_will_not_be.html

Source Link
Gabe K
  • 6k
  • 2
  • 31
  • 60

With regard to your first question, one such result shows that under some conditions on $q$, the following inequality holds for all p:

$W_2(p,q) \leq \sqrt{\frac{ KL(p||q)}{\rho}}$

This is a so called Talagrand$(\rho)$ inequality, which holds whenever a suitable log-Sobolev inequality holds for $q$. For more information, the paper of Otto and Villani [1] is a good reference. This is all contained in Villani's book on optimal transport as well. This also contains many other inequalities between the various distances and divergences, so it is a good reference. However, due to the work of Otto, the Wasserstein 2 metric induces a formal Riemannian metric on the space of densities, and its geometry can be quite different from that of the Fisher-Rao metric.

As to your second question, the Fisher metric is unique in that it is the quadratic term of the Taylor series for any $f$-divergence. For more information on this phenomena, see [2].

[1] Otto, F., & Villani, C. (2000). Generalization of an inequality by Talagrand and links with the logarithmic Sobolev inequality. Journal of Functional Analysis, 173(2), 361-400.

[2] The Fisher Metric Will Not Be Deformed https://golem.ph.utexas.edu/category/2018/05/the_fisher_metric_will_not_be.html