Skip to main content
2 of 4
added 281 characters in body
Iosif Pinelis
  • 127.9k
  • 8
  • 107
  • 229

The answer is no. Indeed, your question can be restated as follows: Is it true that for some constant $c>0$ we have $$\int P\ln\frac PQ\,d\mu\ge c\int(\ln P-\ln Q)^2\,d\mu, \tag{1} $$ where $P$ and $Q$ are probability densities with respect to a measure $\mu$?

Let $\mu$ be the counting measure on the set $\{0,1\}$, and let $P(0)=p\in(0,1)$ and $Q(0)=q\in(0,1)$. Then (1) will become $$p\ln\frac pq+(1-p)\ln\frac{1-p}{1-q}\ge c\Big[\Big(\ln\frac pq\Big)^2+\Big(\ln\frac{1-p}{1-q}\Big)^2\Big]. \tag{2} $$ Letting now $p\to0$, we get a contradiction, because the left-hand side of (2) will go to $\ln\frac1{1-q}<\infty$, but the right-hand side of (2) will go to $\infty$.

Added: You have changed your original question quite a bit, without any further comment, thus just invalidating my answer. Anyhow, the answer now changes from "no" to "obviously no". Indeed, to disprove your latest conjecture, just take any $f,f',g$ such that $f'=f\ne g$.

Iosif Pinelis
  • 127.9k
  • 8
  • 107
  • 229