Given a parametric family of distributions $\{p_\theta|\theta \in \Theta\}$, one can show that under some regularity conditions, the following approximation is valid

$$\operatorname{KL}(p_\theta\| p_{\theta + d \theta}) = d \theta^TF(\theta)d\theta + \mathcal O(\|d\theta\|^3), $$ where $$F(\theta)_{ij} := \mathbb E_{x \sim p_\theta}\left[\frac{\partial^2}{\partial \theta_i\partial \theta_j}\log(p_\theta(x))\right] $$ is the Fisher information matrix of $p_\theta$. A very rough sketch of the proof can be found on wikipedia.

Question 1

Is there such an approximation formula for the Wassertein distance or other measures of discrepancy between probability distributions ?

Question 2

Same question, specialized to $f$-divergences (of which KL is a particular case).

Given a parametric family of distributions $\{p_\theta\mid\theta \in \Theta\}$, one can show that under some regularity conditions, the following approximation is valid

$$\operatorname{KL}(p_\theta\parallel p_{\theta + d \theta}) = d \theta^TF(\theta) \, d\theta + \mathcal O(\|d\theta\|^3), $$ where $$F(\theta)_{ij} := \mathbb E_{x \sim p_\theta}\left[\frac{\partial^2}{\partial \theta_i \, \partial \theta_j} \log(p_\theta(x))\right] $$ is the Fisher information matrix of $p_\theta$. A very rough sketch of the proof can be found on wikipedia.

Question 1

Is there such an approximation formula for the Wassertein distance or other measures of discrepancy between probability distributions ?

Question 2

Same question, specialized to $f$-divergences (of which KL is a particular case).

Given a parametric family of distributions $\{p_\theta|\theta \in \Theta\}$, one can show that under some regularity conditions, the following approximation is valid

$$\operatorname{KL}(p_\theta\| p_{\theta + d \theta}) = d \theta^TF(\theta)d\theta + \mathcal O(\|d\theta\|^3), $$ where $$F(\theta)_{ij} := \mathbb E_{x \sim p_\theta}\left[\frac{\partial^2}{\partial \theta_i\partial \theta_j}\log(p_\theta(x))\right] $$ is the Fisher information matrix of $p_\theta$. A very rough sketch of the proof can be found on wikipedia.

Question

Is there such an approximation formula for the Wassertein distance or other measures of discrepancy between probability distributions ?

Given a parametric family of distributions $\{p_\theta|\theta \in \Theta\}$, one can show that under some regularity conditions, the following approximation is valid

$$\operatorname{KL}(p_\theta\| p_{\theta + d \theta}) = d \theta^TF(\theta)d\theta + \mathcal O(\|d\theta\|^3), $$ where $$F(\theta)_{ij} := \mathbb E_{x \sim p_\theta}\left[\frac{\partial^2}{\partial \theta_i\partial \theta_j}\log(p_\theta(x))\right] $$ is the Fisher information matrix of $p_\theta$. A very rough sketch of the proof can be found on wikipedia.

Question 1

Is there such an approximation formula for the Wassertein distance or other measures of discrepancy between probability distributions ?

Question 2

Same question, specialized to $f$-divergences (of which KL is a particular case).

Given a parametric family of distributions $\{p_\theta|\theta \in \Theta\}$, one can show that under some regularity conditions, the following approximation is valid

$$\operatorname{KL}(p_\theta\| p_{\theta + d \theta}) = d \theta^TF(\theta)d\theta + \mathcal O(\|d\theta\|^3), $$ where $F(\theta) := \mathbb E_{x \sim p_\theta}[\nabla_\theta \log(p_\theta(x))\nabla_\theta \log(p_\theta(x))^T]$ is the Fisher information matrix for $p_\theta$. For example, see this very rough sketch of the proof.

Question

Is there such an approximation formula for the Wassertein distance or other measures of discrepancy between probability distributions ?

Given a parametric family of distributions $\{p_\theta|\theta \in \Theta\}$, one can show that under some regularity conditions, the following approximation is valid

$$\operatorname{KL}(p_\theta\| p_{\theta + d \theta}) = d \theta^TF(\theta)d\theta + \mathcal O(\|d\theta\|^3), $$ where $$F(\theta)_{ij} := \mathbb E_{x \sim p_\theta}\left[\frac{\partial^2}{\partial \theta_i\partial \theta_j}\log(p_\theta(x))\right] $$ is the Fisher information matrix of $p_\theta$. A very rough sketch of the proof can be found on wikipedia.

Question

Is there such an approximation formula for the Wassertein distance or other measures of discrepancy between probability distributions ?