Skip to main content
1 of 4
Turbo
  • 13.9k
  • 1
  • 27
  • 76

Entropy difference dominance of decreasing sequences

Consider a collection of positive integers $\{a_i\}_{i=1}^m$ and the distribution $p_i=\frac{a_i}{\sum_{i=1}^ma_i}$.

Similarly for the collection $\{a_i\}_{i=1}^{m+1}$ form the distribution $q_i=\frac{a_i}{\sum_{i=1}^{m+1}a_i}$, for the collection $\{a_i\}_{i=1}^{n}$ form the distribution $r_i=\frac{a_i}{\sum_{i=1}^{n}a_i}$ and for the collection $\{a_i\}_{i=1}^{n+1}$ form the distribution $s_i=\frac{a_i}{\sum_{i=1}^{n+1}a_i}$.

Let $m+1\leq n$. Note all collections are subsets of the collection for the distribution $s$.

For the cases $(1)$ $a_{i+1}=a_i+\theta(\log^ka_i)$, $(2)$ $a_{i+1}=a_i+\theta(a_i)$ where $k$ is a postive constant, is it true that the difference in entropy is strictly decreasing? That is, is $H(q)-H(p)>H(s)-H(r)$ valid? When can one expect Shannon entropy to be dominant in sequences of these types? Clearly in this post Entropy dominance, one criteria for negative result is given. Can $a_i$ be any larger or smaller than in the cases given?

Turbo
  • 13.9k
  • 1
  • 27
  • 76