# Mutual information between two discrete random variables

I have 2 IID random variables $$X_1$$ and $$X_2$$ with $$Bern(p)$$ distribution. I have another binary random variable $$Y$$ taking values in $$\{0,1\}$$.

I am interested in comparing the following 2 mutual information $$I(X_1+X_2;Y)$$ and $$I(2X_1;Y)$$. Note that $$Y=0$$ with probability $$\frac{1}{x+5}$$ when the input ($$X_1+X_2$$ or $$2X_1$$) takes the value $$x$$.

I have a feeling that $$I(X_1+X_2;Y) \leq I(2X_1;Y)$$. Can someone help me prove or disprove this?

Indeed, the difference
$$d(p):=I(X_1+X_2;Y)-I(2X_1;Y)$$ between $$I(X_1+X_2;Y)$$ and $$I(2X_1;Y)$$ depends only on $$p\in[0,1]$$. The expression for $$d(p)$$ is somewhat complicated, containing a number of logarithmic terms; see the images below of a Mathematica notebook with details of calculations.

However, $$d'''(p)$$ is a rational function of $$p$$, which is rather easy to see to be $$>0$$ on $$(0,1)$$, so that $$d''$$ is increasing on $$[0, 1]$$. Also, $$d''(0)>0$$, so that $$d''>0$$ on $$[0,1]$$ and hence $$d$$ is strictly convex on $$[0, 1]$$. Finally, $$d(0)=d(1)=0$$ and hence $$d<0$$ on $$(0,1)$$.

That is, $$I(X_1+X_2;Y) for $$p\in(0,1)$$ and $$I(X_1+X_2;Y)=I(2X_1;Y)$$ for $$p\in\{0,1\}$$.  • Thank you for the answer. I also feel that this holds in general for any decreasing function of $x$ as well (in the place of $\frac{1}{x+5}$). I am also interested in proving or disproving this. ( I can ask this as a new question if that's better.) Mar 25, 2022 at 2:43
• Yes, I think a new question would be better. Mar 25, 2022 at 3:40
• Thank you sir. I posted a new question which is a more general version of this question. [link] mathoverflow.net/questions/418877/…. Mar 25, 2022 at 4:09