# Lower Bound of KL-Divergence Between Two Gibbs Measures

Suppose we have two Gibbs measures with densities $$p_f(x) \propto \exp(f(x)),\quad q_g(x)\propto \exp(g(x)).$$ Consider the KL-divergence between $$p_f$$ and $$q_g$$, as a functional of $$f$$ and $$g$$, that is, $$D(f, g) := \text{KL}(p_f \| q_g).$$

Question: Do we have the following lower bound: $$D(f, g) \geq \|f - g\|^2,$$ where we are interested in, for example, the $$L_2$$-norm of $$f-g$$.

The answer is no. Indeed, your question can be restated as follows: Is it true that for some constant $$c>0$$ we have $$\int P\ln\frac PQ\,d\mu\ge c\int(\ln P-\ln Q)^2\,d\mu, \tag{1}$$ where $$P$$ and $$Q$$ are probability densities with respect to a measure $$\mu$$?
Let $$\mu$$ be the counting measure on the set $$\{0,1\}$$, and let $$P(0)=p\in(0,1)$$ and $$Q(0)=q\in(0,1)$$. Then (1) will become $$p\ln\frac pq+(1-p)\ln\frac{1-p}{1-q}\ge c\Big[\Big(\ln\frac pq\Big)^2+\Big(\ln\frac{1-p}{1-q}\Big)^2\Big]. \tag{2}$$ Letting now $$p\to0$$, we get a contradiction, because the left-hand side of (2) will go to $$\ln\frac1{1-q}<\infty$$, but the right-hand side of (2) will go to $$\infty$$.