Given a parametric family of distributions $\{p_\theta|\theta \in \Theta\}$, one can show that under some regularity conditions, the following approximation is valid $$\operatorname{KL}(p_\theta\| p_{\theta + d \theta}) = d \theta^TG(\theta)d\theta + \mathcal O(\|d\theta\|^3), $$ where $G(\theta) := \mathbb E_{x \sim p_\theta}[\nabla_\theta \log(p_\theta(x))\nabla_\theta \log(p_\theta(x))^T]$ is the Fisher information matrix for $p_\theta$. For example, see this [very rough sketch of the proof][1]. Question ======== Is there such an approximation formula for the [Wassertein distance][2] or other measures of discrepancy between probability distributions ? [1]: https://en.wikipedia.org/wiki/Fisher_information#Jeffreys_prior_in_Bayesian_statistics [2]: https://en.wikipedia.org/wiki/Wasserstein_metric