Let $H_n : (\langle 0,1 \rangle \cap \mathbb{Q})^n \to \langle 0,\log_2 n \rangle, \; H_n(P) = -\sum_{i=1}^n P_i \log_2 P_i, \; \sum_i P_i = 1$ be the information entropy on rationals. I am looking for an integer-valued analogue $H_n' : \mathbb{N}^n\to\mathbb{Z}$ satisfying the following: Denote $ K_x =\sum_{i=1} C_{x,i}$ then $H_n\left(\frac{C_a}{K_a}\right) > H_n\left(\frac{C_b}{K_b}\right) $ iff $H_n'(C_a) > H_n'(C_b).$ The inequality really needs to be strict. In other words, I am looking for a monotonic transform from $H_n$ to $H_n'$.

I am unable to figure this out, first I thought that putting $H$ to exponent would suffice but $2^{H_n(P)} = \prod_{i=1}^n P_i^{-P_i}$ is obviously not integer valued. Then I ran out of ideas.

  • $\begingroup$ closed interval $\endgroup$ – user1747134 Jun 27 '18 at 10:38

Such a function cannot exist.

Let $I$ be the image of $H_n$. Then $I$ is a countable dense subset of $[0, \log_2 n]$. If such a function $H_n'$ would exist, then $H_n'(C_a) = f\big(H_n\big(\tfrac{C_a}{K_a}\big)\big)$, where $f$ is an injective monotonic function $f:I\to\mathbb{Z}$. Such a function $f$ cannot exist: For any $x,y\in I$ there is $z\in I$ with $x < z < y$, but the same is not true in $\mathbb{Z}$.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.