2
$\begingroup$

I have 2 IID random variables $X_1$ and $X_2$ with $Bern(p)$ distribution. I have another binary random variable $Y$ taking values in $\{0,1\}$.

I am interested in comparing the following 2 mutual information $I(X_1+X_2;Y)$ and $I(2X_1;Y)$. Note that $Y=0$ with probability $\frac{1}{x+5}$ when the input ($X_1+X_2$ or $2X_1$) takes the value $x$.

I have a feeling that $I(X_1+X_2;Y) \leq I(2X_1;Y)$. Can someone help me prove or disprove this?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

Your feeling is correct.

Indeed, the difference
$$d(p):=I(X_1+X_2;Y)-I(2X_1;Y)$$ between $I(X_1+X_2;Y)$ and $I(2X_1;Y)$ depends only on $p\in[0,1]$. The expression for $d(p)$ is somewhat complicated, containing a number of logarithmic terms; see the images below of a Mathematica notebook with details of calculations.

However, $d'''(p)$ is a rational function of $p$, which is rather easy to see to be $>0$ on $(0,1)$, so that $d''$ is increasing on $[0, 1]$. Also, $d''(0)>0$, so that $d''>0$ on $[0,1]$ and hence $d$ is strictly convex on $[0, 1]$. Finally, $d(0)=d(1)=0$ and hence $d<0$ on $(0,1)$.

That is, $I(X_1+X_2;Y)<I(2X_1;Y)$ for $p\in(0,1)$ and $I(X_1+X_2;Y)=I(2X_1;Y)$ for $p\in\{0,1\}$.

enter image description here

enter image description here

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thank you for the answer. I also feel that this holds in general for any decreasing function of $x$ as well (in the place of $\frac{1}{x+5}$). I am also interested in proving or disproving this. ( I can ask this as a new question if that's better.) $\endgroup$
    – wanderer
    Commented Mar 25, 2022 at 2:43
  • 1
    $\begingroup$ Yes, I think a new question would be better. $\endgroup$
    – Iosif Pinelis
    Commented Mar 25, 2022 at 3:40
  • $\begingroup$ Thank you sir. I posted a new question which is a more general version of this question. [link] mathoverflow.net/questions/418877/…. $\endgroup$
    – wanderer
    Commented Mar 25, 2022 at 4:09

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .