Return to Answer

$\newcommand\ep\varepsilon$Of course not.

E.g., suppose that $P(X=0,Y=0)=t\ep$, $P(X=0,Y=1)=1/2-t\ep$, $P(X=1,Y=0)=(1-t)\ep$, and $P(X=1,Y=1)=1/2-(1-t)\ep$, where $t$ and $\ep$ are in the interval $(0,1)$. Then $$I(X;Y)\sim c_t\ep\to0$$ as $\ep\downarrow0$, where $c_t:=\ln2+t\ln t+(1-t)\ln(1-t)$.

However, here $P(Y=0|X=0)=2t\ep$, which is not relatively close to $P(Y=0)=\ep$ if (say) $t=1/4$. (Or you can let $t\downarrow0$; then $c_t\to\ln2$.)

The idea of this counterexample is quite simple: it is to let the conditional probability mass function (pmf) of $Y$ differ substantially from its unconditional pmf on a set of values unlikely to be taken by $Y$.

$\newcommand\ep\varepsilon$Of course not.

E.g., suppose that $P(X=0,Y=0)=t\ep$, $P(X=0,Y=1)=1/2-t\ep$, $P(X=1,Y=0)=(1-t)\ep$, and $P(X=1,Y=1)=1/2-(1-t)\ep$, where $t$ and $\ep$ are in the interval $(0,1)$. Then $$I(X;Y)\sim c_t\ep\to0$$ as $\ep\downarrow0$, where $c_t:=\ln2+t\ln t+(1-t)\ln(1-t)$.

However, here $P(Y=0|X=0)=2t\ep$, which is not relatively close to $P(Y=0)=\ep$ if (say) $t=1/4$. (Or you can let $t\downarrow0$; then $c_t\to\ln2$.)

The idea of this counterexample is quite simple: it is to let the conditional probability mass function (pmf) of $Y$ given $X$ differ substantially from its unconditional pmf only on a set of values rather unlikely to be taken by $Y$. This will affect the mutual information only little.

$\newcommand\ep\varepsilon$Of course not.

E.g., suppose that $P(X=0,Y=0)=t\ep$, $P(X=0,Y=1)=1/2-t\ep$, $P(X=1,Y=0)=(1-t)\ep$, and $P(X=1,Y=1)=1/2-(1-t)\ep$, where $t$ and $\ep$ are in the interval $(0,1)$. Then $$I(X;Y)\sim c_t\ep\to0$$ as $\ep\downarrow0$, where $c_t:=\ln2+t\ln t+(1-t)\ln(1-t)$.

However, here $P(Y=0|X=0)=2t\ep$, which is not relatively close to $P(Y=0)=\ep$ if (say) $t=1/4$.

The idea of this counterexample is quite simple: it is to let the conditional probability mass function (pmf) of $Y$ differ substantially from its unconditional pmf on a set of values unlikely to be taken by $Y$.

$\newcommand\ep\varepsilon$Of course not.

E.g., suppose that $P(X=0,Y=0)=t\ep$, $P(X=0,Y=1)=1/2-t\ep$, $P(X=1,Y=0)=(1-t)\ep$, and $P(X=1,Y=1)=1/2-(1-t)\ep$, where $t$ and $\ep$ are in the interval $(0,1)$. Then $$I(X;Y)\sim c_t\ep\to0$$ as $\ep\downarrow0$, where $c_t:=\ln2+t\ln t+(1-t)\ln(1-t)$.

However, here $P(Y=0|X=0)=2t\ep$, which is not relatively close to $P(Y=0)=\ep$ if (say) $t=1/4$. (Or you can let $t\downarrow0$; then $c_t\to\ln2$.)

The idea of this counterexample is quite simple: it is to let the conditional probability mass function (pmf) of $Y$ differ substantially from its unconditional pmf on a set of values unlikely to be taken by $Y$.

$\newcommand\ep\varepsilon$Of course not.

E.g., suppose that $P(X=0,Y=0)=t\ep$, $P(X=0,Y=1)=1/2-t\ep$, $P(X=1,Y=0)=(1-t)\ep$, and $P(X=1,Y=1)=1/2-(1-t)\ep$, where $t$ and $\ep$ are in the interval $(0,1)$. Then $$I(X;Y)\sim c_t\ep\to0$$ as $\ep\downarrow0$, where $c_t:=\ln2+t\ln t+(1-t)\ln(1-t)$.

However, here $P(Y=0|X=0)=2t\ep$, which is not relatively close to $P(Y=0)=\ep$ if (say) $t=1/4$.

$\newcommand\ep\varepsilon$Of course not.

E.g., suppose that $P(X=0,Y=0)=t\ep$, $P(X=0,Y=1)=1/2-t\ep$, $P(X=1,Y=0)=(1-t)\ep$, and $P(X=1,Y=1)=1/2-(1-t)\ep$, where $t$ and $\ep$ are in the interval $(0,1)$. Then $$I(X;Y)\sim c_t\ep\to0$$ as $\ep\downarrow0$, where $c_t:=\ln2+t\ln t+(1-t)\ln(1-t)$.

However, here $P(Y=0|X=0)=2t\ep$, which is not relatively close to $P(Y=0)=\ep$ if (say) $t=1/4$.

The idea of this counterexample is quite simple: it is to let the conditional probability mass function (pmf) of $Y$ differ substantially from its unconditional pmf on a set of values unlikely to be taken by $Y$.