# Metric ($f$-divergence) on space of probability measures that satisfies pythagorean theorem

Let $$E$$ be a polish space, $$\mathcal{P}(E)$$ the Borel probability measures on $$E$$ with the topology of weak convergence and $$\mathcal{Q} \subset \mathcal{P}(E)$$ a convex and compact set.

First, the motivation for my question:

The Kullback Leibler divergence (or relative entropy) $$R : \mathcal{P}(E)\times \mathcal{P}(E) \rightarrow [0, \infty]$$ is defined by $$R(\nu, \mu) = \int \frac{d\nu}{d\mu} \log(\frac{d\nu}{d\mu}) d\mu$$ if $$\nu$$ is absolutely continuous with respect to $$\mu$$ and $$R(\nu, \mu) = \infty$$, else.

Let $$\theta \in \mathcal{P}(E)$$ and $$\theta^{\ast}_R := \arg\min_{\mu \in \mathcal{Q}} R(\mu, \theta)$$ be the information projection of $$\theta$$ onto $$\mathcal{Q}$$. As a consequence of the pythagorean like theorem for relative entropy (c.f. Csiszár (1975), Theorem 2.2) it holds $$$$\label{eq1} \forall \mu \in \mathcal{Q} : R(\theta^{\ast}_R, \theta) + R(\mu, \theta^{\ast}_R) \leq R(\mu, \theta)$$$$

My question is whether there exists a metric $$d$$ on $$\mathcal{P}(E)$$ which satisfies the same kind of result. So if $$\theta^{\ast}_d := \arg\min_{\mu\in\mathcal{Q}} d(\mu, \theta)$$, it should hold $$\forall \mu \in \mathcal{Q}: d(\theta^{\ast}_d, \theta)^2 + d(\mu, \theta^{\ast}_d)^2 \leq d(\mu, \theta)^2$$

Ideally, the metric $$d$$ should metrize weak convergence, but I am thankful for any approaches or pointers.

• Just as a note, the KL divergence doesn't induce the weak topology (and there are actually two separate topologies depending on whether you take open balls with respect to the first or second index). Similarly, $f$- divergences will also not induce the weak topology. Dec 19, 2019 at 19:38