1
$\begingroup$

Let $X,Y$ be continuous random variables with $X$ defined over $\mathcal{A}$, and let $f: \mathcal{A} \to \mathcal{A}$, $g: \mathcal{A} \to \mathcal{A}$ be any functions. Is it true that

$$ I(Y;f \circ g(X) ) \leq I(Y; f(X)) $$

where $I(\cdot\,; \cdot)$ denotes mutual information. Note that data processing inequality gives, $$ I(Y; f\circ g(X)) \leq I(Y; g(X)) \leq I(Y; X)\\ I(Y; f(X)) \leq I(Y;X) $$

Improve this question
$\endgroup$
3
  • 2
    $\begingroup$ Do you mean $g(X)$ on the right-hand side? Otherwise, you may have domain problems that answer your question immediately - take $X$ supported on $[0, 1]$, $g$ shifting by 1, and $f$ which flattens $[0, 1]$ and is the identity on $[1, 2]$. $\endgroup$
    – user44191
    Commented Oct 27, 2016 at 4:35
  • $\begingroup$ @user44191 If g is on the RHS, it's trivial. You should write it as an answer! $\endgroup$
    – Memming
    Commented Oct 27, 2016 at 13:39
  • $\begingroup$ @user44191, no I am asking with $f(X)$ on the right-hand side. You are right for the domain problems, so let us add that $g:\mathcal{A} \to \mathcal{A}$ and $f:\mathcal{A} \to \mathcal{A}$ $\endgroup$
    – user83947
    Commented Oct 27, 2016 at 15:10

1 Answer 1

Reset to default
2
$\begingroup$

There is no reason for that. For instance, one can choose the functions $f$ and $g$ in such a way that $g$ is a bijection on the range of $X$, whereas $f$ is constant on the range of $X$ and is a bijection on the range of $g(X)$. Then $I(Y,f(X))=0$, whereas $I(Y,f\circ g(X))=I(Y,X)$.

Improve this answer
$\endgroup$

You must log in to answer this question.