Entropy of a refinement of a partition

Question

We consider a probability space $(X, B, \mu)$. Let $\alpha$ and $\beta$ be countable partitions of X. We suppose $\beta$ is a refinement of $\alpha$, ie that every set in $\alpha$ is a union of sets in $\beta$. I am interested into the difference in entropy induced by the partitions $\alpha$ and $\beta$. We call $\cal{H}$ the entropy function.

Then, we know:

${\cal{H}}(\alpha, X) \leq {\cal{H}}(\beta, X)$.

Could we have an evaluation of their difference?

Answer 1

There is a general formula $$ H(\alpha \vee \beta) = H(\alpha)+H(\beta|\alpha) . $$ In your case $\alpha\vee\beta=\beta$, whence $H(\beta)-H(\alpha)$ coincides with the conditional entropy $H(\beta|\alpha)$.