We consider the f-divergence, which takes the form $$ D_f(P \| Q) = \int_\Omega f\left(\frac{dP}{dQ}\right) dQ. $$ For example, when $f(t) = t \log t$, we obtain the KL-divergence.

My question is that, as a function of $P$, under what regularity conditions on $f$ and $Q$ is the f-divergence $D_f(P \| Q)$ (geodesically) $\mu$-smooth with respect to the Wasserstein metric $W_2$?

For any function $g(P)$, the (geodesic) $\mu$-smoothness of $g(P)$ takes the form $$ g(P') \leq g(P) + \langle\text{grad}\,g(P), \exp_P^{-1} (P') \rangle_P + \frac{\mu}{2}\cdot W_2^2(P, P') $$ or equivalently that the Hessian $\text{Hess}(g)$ is bounded from the above by $\mu$ for all $P$. Here, $\text{grad}$ and $\text{Hess}$ are the gradient and Hessian with respect to the $W_2$ metric, which is the geodesic distance in the space of probability distributions, and $\exp_P^{-1} (P')$ is the inverse of the exponential map.


Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.