Given $P(X, Y, \hat{Y})$ discrete with $\hat{Y}$ independent of both $X$ and $Y$, one would thus expect that the following relationship holds $$ \max_{f}I(X;Y,\hat{Y} \mid f(Y,\hat{Y})) = \max_{f_1, f_2}I(X;Y,\hat{Y} \mid f_1(Y) f_2(\hat{Y})) = \max_{f_1}I(X;Y \mid f_1(Y)) $$ where $f, f_1, f_2$ are deterministic, $f$ is non injective and either $f_1$ or $f_2$ is also non-injective functions. Is it the case?
2
-
$\begingroup$ Can you recall what the notation $I(U ; V | W)$ means? $\endgroup$– Christophe LeuridanCommented Dec 3, 2022 at 8:53
-
$\begingroup$ Conditional mutual information $\endgroup$– CesareCommented Dec 3, 2022 at 16:51
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Observe that for any $f(y, \hat{y})$ we have $I(X;Y,\hat{Y}|f(Y,\hat{Y})) \leq I(X;Y,\hat{Y},f(Y,\hat{Y})) = I(X;Y,\hat{Y}) = I(X;Y)$. Equality is achieved by taking $f$ to be a constant, hence this is the maximum value.
Similarly, for any $f_1(y)$ we have $I(X;Y|f_1(Y)) \leq I(X;Y)$. Equality is achieved by taking $f_1$ to be a constant, hence this is the maximum value.
Unless I am misreading your question, the equality holds rather immediately.
