Suppose $p_1,p_2,\dots,p_n \in [0,1]$ and they satisfies $$ \sum_{j=1}^n p_j = 1 $$ and $$ \sum_{j=1}^n p_j^2 = C $$ with a given constant $C \in [1/n,1]$. The problem is to find the minimum of $$ -\sum_{j=1}^n p_j \log p_j .$$ Obviously, this problem could be solved by method of Lagrange multipliers. Since I believe that this problem is pretty typical, I have a question:

Is there any literature (paper or book) which solved this problem?

  • $\begingroup$ It should not be so different from maximum entropy distributions: en.wikipedia.org/wiki/Maximum_entropy_probability_distribution The minimum gets abritrarily close to 0 as $C \to 1$. $\endgroup$ – Campello Aug 2 '17 at 10:24
  • $\begingroup$ Note that you are trying to minimize a concave function, i.e, maximizing a convex functoin. This is difficult in general; your feasible set is also not convex but you can get around that issue. $\endgroup$ – Arash Mar 2 '18 at 7:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.