2
$\begingroup$

For a given mean $\mu$, what is the entropy maximizing probability distribution on the nonnegative integers?

Different sources indicated either the geometric or the Poisson distribution for this. As I am new to the topic, it would be great if someone would give me a hint or point me to a source with a thorough explanation of these issues.

$\endgroup$
6
$\begingroup$

maximizing $S=\sum_{n=0}^\infty p_n \log p_n$ with the constraint $\sum_n n p_n=\mu$ and $\sum_n p_n =1$ gives $p_n = a b^n$, with Lagrange multipliers $a=1/(\mu+1)$ and $b=\mu/(\mu+1)$ determined by the constraints, so this is indeed a geometric distribution.

what is the source you are referring to that says the distribution is Poisson?

$\endgroup$
5
  • $\begingroup$ Just from the net. Probably because Poisson processes are often used to model the situation described above. $\endgroup$ Jan 24 '15 at 13:36
  • $\begingroup$ Hi, Actually there is a source for this "Binomial and Poisson Distributions as Maximum Entropy Distributions" which gives a proof that the maxent distribution for n going to infinity is poisson $\endgroup$
    – Pushpendre
    Jul 20 '15 at 19:47
  • 1
    $\begingroup$ EDIT : the above paper gives a proof that the binomial/poisson are maxent distribution amongst those distributions that are "n-generalized binomial distributions". In other words the binomial and poisson distribution are discrete maxent distributions with a given mean but only among the set of distributions which can be modeled as "n-generalized binomial". See the paper for the meaning of the term. $\endgroup$
    – Pushpendre
    Jul 20 '15 at 19:55
  • $\begingroup$ @Carlo Beenakker I ve been trying to do the calculation on my own. But I am not having much success in getting a=1/u+1 Using lagrange multiplier I’ve gotten up till Summation (pk) -1 =0 Summation (k * pk)- u=0 And -1-log(pk)=a*(1)+b*(k) $\endgroup$ Feb 13 '18 at 17:39
  • $\begingroup$ @Math.love.. --- These are the equations to solve for $a,b$ with $p_n=ab^n$: $\sum_n p_n = a/(1-b)=1$, $\sum_n np_n=ab/(1-b)^2=\mu$; the solution is $a=1/(\mu+1)$ and $b=\mu/(\mu+1)$. $\endgroup$ Feb 13 '18 at 20:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.