8
$\begingroup$

Suppose we have two Gibbs measures with densities $$ p_f(x) \propto \exp(f(x)),\quad q_g(x)\propto \exp(g(x)). $$ Consider the KL-divergence between $p_f$ and $q_g$, as a functional of $f$ and $g$, that is, $$ D(f, g) := \text{KL}(p_f \| q_g). $$

Question: Do we have the following lower bound: $$ D(f, g) \geq \|f - g\|^2, $$ where we are interested in, for example, the $L_2$-norm of $f-g$.

$\endgroup$

1 Answer 1

5
$\begingroup$

The answer is no. Indeed, your question can be restated as follows: Is it true that for some constant $c>0$ we have $$\int P\ln\frac PQ\,d\mu\ge c\int(\ln P-\ln Q)^2\,d\mu, \tag{1} $$ where $P$ and $Q$ are probability densities with respect to a measure $\mu$?

Let $\mu$ be the counting measure on the set $\{0,1\}$, and let $P(0)=p\in(0,1)$ and $Q(0)=q\in(0,1)$. Then (1) will become $$p\ln\frac pq+(1-p)\ln\frac{1-p}{1-q}\ge c\Big[\Big(\ln\frac pq\Big)^2+\Big(\ln\frac{1-p}{1-q}\Big)^2\Big]. \tag{2} $$ Letting now $p\to0$, we get a contradiction, because the left-hand side of (2) will go to $\ln\frac1{1-q}<\infty$, but the right-hand side of (2) will go to $\infty$.

$\endgroup$
0

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .