Reverse Pinsker's inequality for smooth density classes

Suppose we are given a class of probability density functions $$\mathcal{F}$$ so that for every $$f \in \mathcal{F}$$ we have $$\alpha \leq f \leq \beta$$ for some positive $$\alpha, \beta \in \mathbb{R}_+$$ and the density have the same support set $$B$$.

Here it is proved that in this setting the $$L_2$$ norm between density functions, i.e., $$\|f-g\|^2_2 = \int_B (f(x) - g(x))^2 \mu(dx)$$ is equivalent to the KL divergence between $$f$$ and $$g$$, i.e., for $$D_{\operatorname{KL}}(f \| g) = \int_B f(x) \log \frac{f(x)}{g(x)} \mu(dx)$$ we have $$D_{\operatorname{KL}}(f \| g) \asymp \|f-g\|^2_2$$ where $$\asymp$$ denotes equiality up to constant factors (which may depend on $$\alpha, \beta$$ but we consider those fixed).

Here a question was asked regarding the comparison of $$L_1$$ and $$L_2$$, and was argued that for such $$\mathcal{F}$$ classes of functions, it is possible that $$\|f-g\|_1 = \int_B |f(x) - g(x)| \mu(dx) \ll \|f-g\|_2$$ in general.

By Pinsker's inequality we know that $$\|f-g\|_1 \lesssim \sqrt{D_{\operatorname{KL}}(f \| g)} \asymp \|f-g\|_2$$ where $$\lesssim$$ indicates inequality up to absolute constant factors.

My question is as follows. Suppose we impose smoothness assumptions on $$\mathcal{F}$$. For instance suppose $$\mathcal{F}$$ consists of Lipschitz continuous functions with Lipschitz constant $$L$$. Does a reverse Pinsker's inequality hold in this case i.e. is it possible that $$\|f-g\|_1 \gtrsim \sqrt{D_{\operatorname{KL}}(f \| g)} \asymp \|f-g\|_2\;?$$

The counterexample given here fails to work in the Lipschtiz $$\mathcal{F}$$ case, and is not clear how it can be recreated because the functions need to be smooth.

• I've generally seen reverse pinsker inequalities only under assumptions that $c \leq f(x)/g(x)\leq c'$ are bounded for all $x$, see for example this. One can get something sort-of like a reverse pinsker inequality via bounds $D_{\mathsf{KL}(T(f)||T(g))} \leq C_T \lVert f -g\rVert_1^2$, where $T(f)$ is the markov kernel $T$ acting on the random variable with density $f$, and $C_T$ is a constant that depends on $T$, but not $f,g$. See for example this. Not relevant to your exact question, but hopefully useful. Mar 25 at 22:13

E.g., suppose that both $$f$$ and $$g$$ are supported on $$B=[0,1]$$, with $$f=1$$ on $$B$$ and $$g(x)=(1-h+x)\,1(0\le x for $$h\in(0,1/2)$$ and $$x\in B$$.
Then $$f$$ and $$g$$ are $$1$$-Lipschitz on $$B$$, $$\|f-g\|_1=h^2$$, and $$D_{\operatorname{KL}}(f\|g) =-\int_0^h dx\,\ln(1-h+x)-\int_{1-h}^1 dx\,\ln(x+h) \\ =-\int_0^h dx\,\ln(1-(h-x)^2) \ge\int_0^h dx\,(h-x)^2=h^3/3,$$ so that $$\|f-g\|_1\not\gtrsim\sqrt{D_{\operatorname{KL}}(f \| g)}$$ if $$h$$ is small enough.
• Awesome counterexample thanks! Do you have any thoughts on whether the exponents of $h$ can be made even more discrepant? In this example for instance $\|f-g\|_1 = h^2$ while $\|f-g\|_2 = h$. Mar 25 at 20:32