Integrability of $\int \log(f(x)) f(x) dx$ for a probability density function $f$

Question

I am looking for weak conditions when a probability density function $f$ on $\mathbb{R}^d$ has a finite integral

$$ \int_{\mathbb{R}^d} \log(f(x)) f(x) dx. $$

Any references would be appreciated.

Answer 1

$\newcommand\R{\mathbb R}$Let $\int h:=\int_{\mathbb R^d}h(x)\,dx$.

Claim: For $\int f\,\ln f$ to be finite, it is enough that $$f(x)\le\frac C{(e+|x|)^d \ln^a(e+|x|)}\tag{1}$$ for some real $a>2$, some real $C>0$, and all $x\in\R^d$.

The condition $a>2$ here cannot be replaced by $a=2$.

Indeed, suppose that (1) holds with $a>2$. Since $\ln f\le\ln C$ and $\int f=1$, we have $$\int f\,\ln f<\infty.\tag{2}$$

On the other hand, consider the probability density function (pdf) $g$ defined by $$g(x):=\frac{c_d}{(e+|x|)^{d+1}}$$ for some real $c_d>0$ and all $x\in\R^d$. Switching to polar coordinates, we find that $$\int f\,\ln g\in\R.\tag{3}$$ Also, $$\int f\,\ln f-\int f\,\ln g=-\int f\,\ln\frac gf\ge-\int f\,\Big(\frac gf-1\Big) =\int f-\int g\,1(f\ne0)\ge0.$$ Therefore, in view of (3), $$\int f\,\ln f>-\infty.$$ This together with (2) yields $$\int f\,\ln f\in\R,\tag{4}$$ if (1) holds with $a>2$.

On the other hand, if the pdf $f$ is given by
$$f(x)=\frac{K_d}{(e+|x|)^d \ln^2(e+|x|)}\tag{5}$$ for a real $K_d>0$ and all $x\in\R^d$, then it is easy to see that $\int f\,\ln f=-\infty$. So, the condition $a>2$ for (1) cannot be replaced by $a=2$.

The claim is now completely proved.