2
$\begingroup$

For a given mean $\mu$, what is the entropy maximizing probability distribution on the nonnegative integers?

Different sources indicated either the geometric or the Poisson distribution for this. As I am new to the topic, it would be great if someone would give me a hint or point me to a source with a thorough explanation of these issues.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
6
$\begingroup$

maximizing $S=\sum_{n=0}^\infty p_n \log p_n$ with the constraint $\sum_n n p_n=\mu$ and $\sum_n p_n =1$ gives $p_n = a b^n$, with Lagrange multipliers $a=1/(\mu+1)$ and $b=\mu/(\mu+1)$ determined by the constraints, so this is indeed a geometric distribution.

what is the source you are referring to that says the distribution is Poisson?

Improve this answer
$\endgroup$
5
  • $\begingroup$ Just from the net. Probably because Poisson processes are often used to model the situation described above. $\endgroup$
    – J Fabian Meier
    Commented Jan 24, 2015 at 13:36
  • $\begingroup$ Hi, Actually there is a source for this "Binomial and Poisson Distributions as Maximum Entropy Distributions" which gives a proof that the maxent distribution for n going to infinity is poisson $\endgroup$
    – Pushpendre
    Commented Jul 20, 2015 at 19:47
  • 2
    $\begingroup$ EDIT : the above paper gives a proof that the binomial/poisson are maxent distribution amongst those distributions that are "n-generalized binomial distributions". In other words the binomial and poisson distribution are discrete maxent distributions with a given mean but only among the set of distributions which can be modeled as "n-generalized binomial". See the paper for the meaning of the term. $\endgroup$
    – Pushpendre
    Commented Jul 20, 2015 at 19:55
  • $\begingroup$ @Carlo Beenakker I ve been trying to do the calculation on my own. But I am not having much success in getting a=1/u+1 Using lagrange multiplier I’ve gotten up till Summation (pk) -1 =0 Summation (k * pk)- u=0 And -1-log(pk)=a*(1)+b*(k) $\endgroup$
    – Math.love..
    Commented Feb 13, 2018 at 17:39
  • $\begingroup$ @Math.love.. --- These are the equations to solve for $a,b$ with $p_n=ab^n$: $\sum_n p_n = a/(1-b)=1$, $\sum_n np_n=ab/(1-b)^2=\mu$; the solution is $a=1/(\mu+1)$ and $b=\mu/(\mu+1)$. $\endgroup$
    – Carlo Beenakker
    Commented Feb 13, 2018 at 20:32

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .