2
$\begingroup$

I have 2 IID random variables $X_1$ and $X_2$ with $Bern(p)$ distribution. I have another binary random variable $Y$ taking values in $\{0,1\}$.

I am interested in comparing the following 2 mutual information $I(X_1+X_2;Y)$ and $I(2X_1;Y)$. Note that $Y=0$ with probability $\frac{1}{x+5}$ when the input ($X_1+X_2$ or $2X_1$) takes the value $x$.

I have a feeling that $I(X_1+X_2;Y) \leq I(2X_1;Y)$. Can someone help me prove or disprove this?

$\endgroup$

1 Answer 1

2
$\begingroup$

Your feeling is correct.

Indeed, the difference
$$d(p):=I(X_1+X_2;Y)-I(2X_1;Y)$$ between $I(X_1+X_2;Y)$ and $I(2X_1;Y)$ depends only on $p\in[0,1]$. The expression for $d(p)$ is somewhat complicated, containing a number of logarithmic terms; see the images below of a Mathematica notebook with details of calculations.

However, $d'''(p)$ is a rational function of $p$, which is rather easy to see to be $>0$ on $(0,1)$, so that $d''$ is increasing on $[0, 1]$. Also, $d''(0)>0$, so that $d''>0$ on $[0,1]$ and hence $d$ is strictly convex on $[0, 1]$. Finally, $d(0)=d(1)=0$ and hence $d<0$ on $(0,1)$.

That is, $I(X_1+X_2;Y)<I(2X_1;Y)$ for $p\in(0,1)$ and $I(X_1+X_2;Y)=I(2X_1;Y)$ for $p\in\{0,1\}$.


enter image description here

enter image description here

$\endgroup$
3
  • $\begingroup$ Thank you for the answer. I also feel that this holds in general for any decreasing function of $x$ as well (in the place of $\frac{1}{x+5}$). I am also interested in proving or disproving this. ( I can ask this as a new question if that's better.) $\endgroup$
    – wanderer
    Mar 25, 2022 at 2:43
  • 1
    $\begingroup$ Yes, I think a new question would be better. $\endgroup$ Mar 25, 2022 at 3:40
  • $\begingroup$ Thank you sir. I posted a new question which is a more general version of this question. [link] mathoverflow.net/questions/418877/…. $\endgroup$
    – wanderer
    Mar 25, 2022 at 4:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.