Consider discrete random variable $X$ with finite support $\mathcal{X}$, continuous random variable $Y$ with bounded support $\mathcal{Y}$ , and constants $c_1 ,c_2 \in \mathbb{R}$. Let $p_{XY}(x,y)$ be the probability measure on $\mathcal{X} \times \mathcal{Y}$. Define $H(X,Y)$ and $H(Y\mid X)$ as the joint and conditional entropies. Define $Z = c_1 X_1 + c_2 X_2$ where $X_1$ and $X_2$ are i.i.d. realizations from the probbaility measure on $X$. What are $c_1,c_2$ that maximize $H(Y,Z)-H(Y\mid Z)$? what about general case with $n$ constants $c_1,...,c_n$?
Maximizing mutual information between two linearly projected random variables
Jeff
- 482
- 2
- 8