The normal distribution on $\mathbb{R}$, the exponential distribution on $\mathbb{R}_{\geq 0}$, and the geometric distribution on $\mathbb{N}$ are examples of distributions that are both infinitely divisible and entropy maximizers. On the other hand, the Poisson distribution is an infinitely divisible distribution on $\mathbb{N}$ without maximizing entropy, while the uniform distribution on the interval $[a,b]$ maximizes entropy but is not infinitely divisible.

Can anything be said about the relationship between these two classes of distributions?

  • $\begingroup$ I don't see a-priori why there should be some intrinsic relationship between infinite divisibility and max-ent, but maybe i am missing some intuition here. $\endgroup$
    – Suvrit
    Jun 18 '11 at 13:31
  • 2
    $\begingroup$ Doesn't the Poisson distribution maximize entropy with respect to the measure that assigns $1/n!$ to $\{n\}$ for $n \in \{0,1,2,\ldots\}$, among all measures on $\{0,1,2,\ldots\}$ having a given expected value? (BTW, a nice way to see istantly that the uniform distribution on $[0,1]$ is not infinitely divisible is that its fourth cumulant is negative. The even-degree cumulants of infinitely divisible distributions are non-negative.) $\endgroup$ Jun 19 '11 at 16:01

The Poisson distribution has maximum entropy under the condition of having a specific mean and being a sum of Bernoulli random variables. The condition of being a sum of Bernoulli random variables can be weakened to being ultra log-concave.

  • $\begingroup$ Is this still true if one considers sums of integer multiples of Bernoulli random variables? $\endgroup$ Mar 28 '15 at 20:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.