Skip to main content
5 of 5
added 18 characters in body
Michael Hardy
  • 1
  • 12
  • 85
  • 126

Approximation of Wasserstein distance between $p_\theta$ and $p_{\theta + d\theta}$

Given a parametric family of distributions $\{p_\theta\mid\theta \in \Theta\}$, one can show that under some regularity conditions, the following approximation is valid

$$\operatorname{KL}(p_\theta\parallel p_{\theta + d \theta}) = d \theta^TF(\theta) \, d\theta + \mathcal O(\|d\theta\|^3), $$ where $$F(\theta)_{ij} := \mathbb E_{x \sim p_\theta}\left[\frac{\partial^2}{\partial \theta_i \, \partial \theta_j} \log(p_\theta(x))\right] $$ is the Fisher information matrix of $p_\theta$. A very rough sketch of the proof can be found on wikipedia.

Question 1

Is there such an approximation formula for the Wassertein distance or other measures of discrepancy between probability distributions ?

Question 2

Same question, specialized to $f$-divergences (of which KL is a particular case).

dohmatob
  • 6.9k
  • 1
  • 18
  • 76