Given a discrete random variable $(X,Y)$, one can consider the smallest entropy of a random variable $Z$ such that $X$ and $Y$ are independent conditioned to $Z$.

This quantity is akin to the mutual information, in fact one can see that it is always larger than half the mutual information.

Does this number have a name, and has it been studied?


This is related to Wyner's common information which is always larger than the mutual information.


| cite | improve this answer | |
  • $\begingroup$ thanks this is useful. This seems to be pretty close indeed, even though it's not clear to me if it's exactly the same $\endgroup$ – alesia May 13 '18 at 0:19
  • 2
    $\begingroup$ Starting from this reference, I found that the variant is called exact common information and was introduced in a 2014 paper by El Gamal et al. It is apparently an open problem whether it coincides with Wyner's common information. $\endgroup$ – alesia May 13 '18 at 1:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.