We consider a probability space $(X, B, \mu)$. Let $\alpha$ and $\beta$ be countable partitions of X. We suppose $\beta$ is a refinement of $\alpha$, ie that every set in $\alpha$ is a union of sets in $\beta$. I am interested into the difference in entropy induced by the partitions $\alpha$ and $\beta$. We call $\cal{H}$ the entropy function.

Then, we know:

${\cal{H}}(\alpha, X) \leq {\cal{H}}(\beta, X)$.

Could we have an evaluation of their difference?

  • $\begingroup$ What do you mean here by "an evaluation"? $\endgroup$ – Iosif Pinelis Oct 7 '20 at 14:14

There is a general formula $$ H(\alpha \vee \beta) = H(\alpha)+H(\beta|\alpha) . $$ In your case $\alpha\vee\beta=\beta$, whence $H(\beta)-H(\alpha)$ coincides with the conditional entropy $H(\beta|\alpha)$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.