Skip to main content

An inequality of KL Divergence for two different distributions passing through a same channel

Let $X\in\mathcal{X}$ be a random variable. Assume that we pass $X$ through two independent channels $p_{X_1|X}$ and $p_{X_1|X}$ and choose $X_1$ with probability $\lambda$ and $X_2$ with $\bar{\lambda}$. \begin{align} Z=\begin{cases} X_1&\text{with prob.}~ \lambda\\ X_2&\text{with prob.}~ \bar{\lambda}. \end{cases} \end{align} Therefore, the resulted probability density function of $Z$ is $\lambda p_{X_1}+\bar{\lambda}p_{X_2}$. Assume that $X_1,X_2$ also belong to $\mathcal{X}$. Now assume that both $X$ and $Z$ cross from a conditional pdf $p_{Y|X}$ and produce random variables $Y_1$ and $Y_2$ respectively. How one can find $\lambda$ such that the following inequality for KL divergrnce is satisfied \begin{align} \mathrm{D}(p_{Y|Z}||p_{Y_2})\leq \mathrm{D}(p_{Y|X}||p_{Y_1}). \end{align} Is it possible?

Math_Y
  • 287
  • 2
  • 16