Let $X$ be a random variable which takes values in $\mathcal{X}$. Assume that we pass $X$ through two independent conditional pdf $p_{X_1|X}$ and $p_{X_2|X}$ and choose $X_1$ with probability $\lambda$ and $X_2$ with $\bar{\lambda}=1-\lambda$. Then, we have the following random variable $Z$: \begin{align} Z=\begin{cases} X_1&\text{with prob.}~ \lambda\\ X_2&\text{with prob.}~ \bar{\lambda}. \end{cases} \end{align} Therefore, the resulted probability density function of $Z$ is $\lambda p_{X_1}+\bar{\lambda}p_{X_2}$. Assume that $X_1,X_2$ also belong to $\mathcal{X}$. Now assume that both $X$ and $Z$ experience a conditional pdf $p_{Y|X}$ and produce random variables $Y_1$ and $Y_2$ respectively, i.e., \begin{align} p_{Y_1}&=\sum_{x\in\mathcal{X}}p_{Y|X=x}p_X(x)\\ p_{Y_2}&=\sum_{x\in\mathcal{X}}p_{Y|Z=x}p_Z(z). \end{align} How one can find $\lambda$ such that the following inequality for KL divergrnce is satisfied \begin{align} \mathrm{D}(p_{Y|Z}||p_{Y_2})\leq \mathrm{D}(p_{Y|X}||p_{Y_1}). \end{align} Is it possible?
$\begingroup$
$\endgroup$
4
-
3$\begingroup$ Can you restate your question in formal mathematical terms, without using such terms as "channels", "cross", and "produce"? Also, even though one could guess what you mean by $\bar\lambda$, can you still define this symbol? Also, by $X\in\mathcal X$ you apparently mean "$X$ takes values in $\mathcal X$", which is of course quite different from $X\in\mathcal X$. $\endgroup$– Iosif PinelisCommented Sep 2, 2020 at 15:58
-
$\begingroup$ Math_Y, your definitions are too lax. Nothing about X1, X2 enforces that Z resemble or preserves information about X in any way. This leaves the two sides of your desired inequality unrelated. $\endgroup$– Christian ChapmanCommented Mar 18, 2021 at 14:59
-
$\begingroup$ What's more, as you've written it, the LHS $D(P_{Y|Z}|P_{Y_2})$ is going to be a random variable depending on $Z$ and the RHS is a RV depending on $X$. I'm not sure that this is what you are really after. $\endgroup$– Christian ChapmanCommented Mar 18, 2021 at 15:09
-
$\begingroup$ I'm guessing that you're trying to formalize a notion similar to this: $${}$$ "$X_1$ and $X_2$ are like $X$, but with different types of noise. $Z$ is a mixing of $X_1$ and $X_2$. There is also some process $Y(\cdot)$ that acts on RVs that live in $\mathcal{X}$. Can I always mix $Z$ so that I learn less about $Y(Z)$ from $Z$ than I learn about $Y(X)$ from $X$?". $${}$$ The answer seems like yes, but what you've written isn't quite the right formalization yet. In particular you need to more closely specify the nature of $X_1$ and $X_2$ (and more cleanly specify the entire problem). $\endgroup$– Christian ChapmanCommented Mar 18, 2021 at 15:21
Add a comment
|