Skip to main content
3 of 5
added 28 characters in body
dohmatob
  • 6.9k
  • 1
  • 18
  • 76

Approximation of Wasserstein distance between $p_\theta$ and $p_{\theta + d\theta}$

Given a parametric family of distributions $\{p_\theta|\theta \in \Theta\}$, one can show that under some regularity conditions, the following approximation is valid

$$\operatorname{KL}(p_\theta\| p_{\theta + d \theta}) = d \theta^TF(\theta)d\theta + \mathcal O(\|d\theta\|^3), $$ where $$F(\theta)_{ij} := \mathbb E_{x \sim p_\theta}\left[\frac{\partial^2}{\partial \theta_i\partial \theta_j}\log(p_\theta(x))\right] $$ is the Fisher information matrix of $p_\theta$. A very rough sketch of the proof can be found on wikipedia.

Question

Is there such an approximation formula for the Wassertein distance or other measures of discrepancy between probability distributions ?

dohmatob
  • 6.9k
  • 1
  • 18
  • 76