Consider a discrete trivariate distribution $P(X_1, X_2, Y)$, which satisfies $$ p(x_1, x_2, y) = \min( p(x_1,y), p(x_2,y) ), $$ for all $x_1$ and $x_2$ for which $p(x_1, x_2) > 0$ and for all values of $y$ (including those at which $p(x_1, x_2, y) = 0$). I am trying to show that then the interaction-information is non-negative $$ I(X_1;Y) + I(X_2;Y) - I(X_1 X_2;Y) \ge 0 $$

I need either a rigorous proof or, alternatively, a counterexample that disproves the theorem.

Addendum 1: I use the shorthand notation $p(x_1, x_2, y) := P(X_1=x_1, X_2 = x_2, Y=y)$

Addendum 2: for a similar inequality and its proof see here. This might provide some inspiration. For theoretical references on non-Shannon inequalities see here and here.

Addendum 3: Note that the canonical example of negative interaction-information, the XOR gate, does not satisfy the theorem hypothesis. For the XOR gate $p(x_1, x_2, y) = \min \left( \ p(x_1,y), \ p(x_2,y) \ \right)$ holds for all $x_1, x_2, y$ such that $p(x_1, x_2, y)>0$ but not for all $y$ once we pick $x_1, x_2$ points for which $p(x_1, x_2)>0$, i.e. also the $y$s for which $p(x_1, x_2, y)=0$.

  • $\begingroup$ I am checking whether the approach from here can also be applied to this problem. $\endgroup$
    – Cesare
    Aug 14 '19 at 7:07
  • $\begingroup$ OP has raised a related question, mathoverflow.net/questions/338330/… $\endgroup$ Aug 14 '19 at 13:09
  • $\begingroup$ Yes, actually that is my question. Since I can't prove the theorem I am starting to wonder whether the satement is wrong. $\endgroup$
    – Cesare
    Aug 14 '19 at 13:11
  • $\begingroup$ I know it's your question, Cesare – OP means original poster. I just thought anyone seeing this question would want to know about the other one. $\endgroup$ Aug 14 '19 at 13:13
  • $\begingroup$ Sorry, didn't know the meaning of OP $\endgroup$
    – Cesare
    Aug 14 '19 at 13:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.