# Data processing inequality

Let $X,Y$ be continuous random variables with $X$ defined over $\mathcal{A}$, and let $f: \mathcal{A} \to \mathcal{A}$, $g: \mathcal{A} \to \mathcal{A}$ be any functions. Is it true that

$$I(Y;f \circ g(X) ) \leq I(Y; f(X))$$

where $I(\cdot\,; \cdot)$ denotes mutual information. Note that data processing inequality gives, $$I(Y; f\circ g(X)) \leq I(Y; g(X)) \leq I(Y; X)\\ I(Y; f(X)) \leq I(Y;X)$$

• Do you mean $g(X)$ on the right-hand side? Otherwise, you may have domain problems that answer your question immediately - take $X$ supported on $[0, 1]$, $g$ shifting by 1, and $f$ which flattens $[0, 1]$ and is the identity on $[1, 2]$. – user44191 Oct 27 '16 at 4:35
• @user44191 If g is on the RHS, it's trivial. You should write it as an answer! – Memming Oct 27 '16 at 13:39
• @user44191, no I am asking with $f(X)$ on the right-hand side. You are right for the domain problems, so let us add that $g:\mathcal{A} \to \mathcal{A}$ and $f:\mathcal{A} \to \mathcal{A}$ – user297646 Oct 27 '16 at 15:10

There is no reason for that. For instance, one can choose the functions $f$ and $g$ in such a way that $g$ is a bijection on the range of $X$, whereas $f$ is constant on the range of $X$ and is a bijection on the range of $g(X)$. Then $I(Y,f(X))=0$, whereas $I(Y,f\circ g(X))=I(Y,X)$.