0
$\begingroup$

Consider the discrete random variables $X,Y,Z,W$, where $Y$ and $W$ are independent. Let $Y = f(X)$, for some function $f(\cdot)$, $Z=Y+W$, and $X \rightarrow Y \rightarrow Z$ be a Markov chain.

By definition, the mutual information satisfies $I(Y;Z) = H(Y) - H(Y|Z) = H(Z) - H(Z|Y)$.

I know that because $Z=Y+W$ and $Y$ and $W$ are independent, then $H(Z|Y) = H(Y+W|Y) = H(W|Y) = H(W)$.

However, I found someone claiming in a paper that $H(Y|Z) = H(Z-W|Z) = H(W|Z) = H(W)$. I cannot understand this because $W$ and $Z$ are correlated. So why is it true that $H(W|Z) = H(W)$? Is it because of the Markov chain assumption?

Any insight would be greatly appreciated! Thanks for reading me.

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ You're right. If $Y$ takes the values 0 and 2 with equal probability and $W$ takes the values 0 and 1 with equal probability, independently of $Y$, then $H(Y|Z)$ is 0 since $Z$ determines $Y$. $\endgroup$
    – Anthony Quas
    Mar 6, 2018 at 17:40
  • $\begingroup$ To the above, in your assumptions, $H(Y|Z)$ is $0$ but $H(W)=1$, wouldn't that contradict $H(Y|Z) = H(W)$? $\endgroup$
    – Lazy Lee
    Mar 14, 2018 at 2:33

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.