1
$\begingroup$

Let $X$ be an absolutely continuous (i.e. its law is absolutely continuous with respect to the Lebesgue measure) random variable with probability density $p$. Its differential entropy is given by $$h(X) = - \int_{\mathbb{R}} p(x) \log p(x) \mathrm{d} x$$ with the convention $0 \log 0 = 0$, as soon as the integral is absolutely convergent.

A random variable is infinitely divisible if, for any $n \geq 1$, $X$ can be decomposed as the sum of $n$ i.i.d. random variables.

Question: Are there infinitely divisible and absolutely continuous random variables for which the differential entropy does not exist?

Comment: It is possible to construct random variables for which the differential entropy does not exist. The constructions I could find are however handcrafted to make the differential entropy undefined. Since infinitely divisible random variables have a strong structure, I am wondering what can be said in this case.

It is moreover possible to find simple conditions so that the differential entropy is well-defined, for instance if $X$ admits some positive moments and $p$ is a bounded probability density. The former condition is however not always true for infinitely divisible laws, and I have no idea for the latter.

Any help would be appreciated.

$\endgroup$

1 Answer 1

2
$\begingroup$

For real $t>0$, let \begin{equation} p_t:=e^{-t}e^{*tf}*g_t:=e^{-t}\sum_{n=0}^\infty\frac{t^n f^{*n}}{n!}*g_t, \tag{0} \end{equation} where $f$ is the (bounded by $c:=1/e$) pdf given by
\begin{equation} f(x)=\frac{1\{x\ge e\}}{x\ln^2 x}, \tag{0.5} \end{equation} $f^{*n}:=f*\cdots*f$ ($n$ times, with $f^{*0}$ defined as the Dirac delta function at $0$), and $g_t$ is the normal pdf with mean $0$ and variance $t$. Then $p_s*p_t=p_{s+t}$ for all real $s,t>0$. So, $$p:=p_1$$ is the pdf of an infinitely divisible distribution. Moreover, similarly to the main result in this paper, we have \begin{equation} p(x)\sim f(x) \tag{1} \end{equation} (the convergence everywhere here is as $x\to\infty$), whence \begin{equation} p(x)\ln p(x)\sim-\frac1{x\ln x}, \tag{1.5} \end{equation} so that the differential entropy does not exist.


Since the proof of (1) is a bit involved, let us make do with something weaker than (1), which however can be proved quickly. Indeed, note first here that, by (0) and (0.5), for $g:=g_1$ and all real $x$ \begin{equation} p(x)\ge e^{-1}(f*g)(x)\ge\frac1e\,\int_{-1}^1f(x-y)g(y)\,dy\sim\frac c{x\ln^2 x}=:q(x), \tag{2} \end{equation} where $c:=\frac1e\,\int_{-1}^1g(y)\,dy\in(0,\infty)$. On the other hand, again by (0), \begin{equation} p(x)=\frac1e\,\int_{-\infty}^\infty g(x-y)\,(e^{*f})(y)\,dy\to0, \tag{3} \end{equation} by dominated convergence. Now note that the function $u\mapsto-u\ln u$ is positive and increasing in a right neighborhood of $0$. Hence, by (2) and (3), for all large enough $x>0$
\begin{equation*} -p(x)\ln p(x)\ge-\frac{q(x)}2\,\ln\frac{q(x)}2\sim\frac c{2x\ln x} \end{equation*} (cf. (1.5)). So, the differential entropy does not exist.

$\endgroup$
11
  • $\begingroup$ Nice! I will definitely keep this way of constructing infinitely divisible laws with given asymptotic behaviors for the pdf. Thanks a lot, this solves my question. $\endgroup$
    – Goulifet
    Commented Jul 15, 2020 at 21:22
  • $\begingroup$ @Goulifet : I am glad this was of help. $\endgroup$ Commented Jul 16, 2020 at 0:25
  • $\begingroup$ @IosifPinelis One can just take the exponential semigroup of any infinite differential entropy measure on $\mathbb R$ without involving the Gaussian semigroup. On the other hand, one can also take the exponential semigroup of an infinite entropy measure on $\mathbb Z$ and then convolve it with the Gaussian semigroup. $\endgroup$
    – R W
    Commented Jul 16, 2020 at 0:54
  • $\begingroup$ @RW : What I used here was indeed the $*$-exponentiation of a finite measure $\mu$. However, one cannot do here without the Gaussian component (even if $\mu$ is absolutely continuous), because otherwise the resulting measure $e^{*\mu}$ would not be absolutely continuous. Anyhow, the main problem here is to prove (1) or its adequately relaxed counterpart. $\endgroup$ Commented Jul 16, 2020 at 1:20
  • $\begingroup$ @ IosifPinelis Sorry - forgot about the 0 order term in the exponent, which gives yet another reason to deal with a distribution on $\mathbb Z$, and then to smooth it. However, I believe that it can be done without using any explicit estimates, just from infinitude of the entropy. $\endgroup$
    – R W
    Commented Jul 16, 2020 at 2:42

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .