Given a discrete random variable $(X,Y)$, one can consider the smallest entropy of a random variable $Z$ such that $X$ and $Y$ are independent conditioned to $Z$.

This quantity is akin to the mutual information, in fact one can see that it is always larger than half the mutual information.

Does this number have a name, and has it been studied?


1 Answer 1


This is related to Wyner's common information which is always larger than the mutual information.


  • $\begingroup$ thanks this is useful. This seems to be pretty close indeed, even though it's not clear to me if it's exactly the same $\endgroup$
    – alesia
    May 13, 2018 at 0:19
  • 2
    $\begingroup$ Starting from this reference, I found that the variant is called exact common information and was introduced in a 2014 paper by El Gamal et al. It is apparently an open problem whether it coincides with Wyner's common information. $\endgroup$
    – alesia
    May 13, 2018 at 1:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.