1
$\begingroup$

Let $X,Y$ be continuous random variables with $X$ defined over $\mathcal{A}$, and let $f: \mathcal{A} \to \mathcal{A}$, $g: \mathcal{A} \to \mathcal{A}$ be any functions. Is it true that

$$ I(Y;f \circ g(X) ) \leq I(Y; f(X)) $$

where $I(\cdot\,; \cdot)$ denotes mutual information. Note that data processing inequality gives, $$ I(Y; f\circ g(X)) \leq I(Y; g(X)) \leq I(Y; X)\\ I(Y; f(X)) \leq I(Y;X) $$

$\endgroup$
3
  • 2
    $\begingroup$ Do you mean $g(X)$ on the right-hand side? Otherwise, you may have domain problems that answer your question immediately - take $X$ supported on $[0, 1]$, $g$ shifting by 1, and $f$ which flattens $[0, 1]$ and is the identity on $[1, 2]$. $\endgroup$ – user44191 Oct 27 '16 at 4:35
  • $\begingroup$ @user44191 If g is on the RHS, it's trivial. You should write it as an answer! $\endgroup$ – Memming Oct 27 '16 at 13:39
  • $\begingroup$ @user44191, no I am asking with $f(X)$ on the right-hand side. You are right for the domain problems, so let us add that $g:\mathcal{A} \to \mathcal{A}$ and $f:\mathcal{A} \to \mathcal{A}$ $\endgroup$ – user297646 Oct 27 '16 at 15:10
3
$\begingroup$

There is no reason for that. For instance, one can choose the functions $f$ and $g$ in such a way that $g$ is a bijection on the range of $X$, whereas $f$ is constant on the range of $X$ and is a bijection on the range of $g(X)$. Then $I(Y,f(X))=0$, whereas $I(Y,f\circ g(X))=I(Y,X)$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.