Skip to main content
Notice removed Draw attention by CommunityBot
Bounty Ended with no winning answer by CommunityBot
edited body
Source Link
Math_Y
  • 287
  • 2
  • 16

Let $X$ be a random variable which takes values in $\mathcal{X}$. Assume that we pass $X$ through two independent conditional pdf $p_{X_1|X}$ and $p_{X_1|X}$$p_{X_2|X}$ and choose $X_1$ with probability $\lambda$ and $X_2$ with $\bar{\lambda}=1-\lambda$. Then, we have the following random variable $Z$: \begin{align} Z=\begin{cases} X_1&\text{with prob.}~ \lambda\\ X_2&\text{with prob.}~ \bar{\lambda}. \end{cases} \end{align} Therefore, the resulted probability density function of $Z$ is $\lambda p_{X_1}+\bar{\lambda}p_{X_2}$. Assume that $X_1,X_2$ also belong to $\mathcal{X}$. Now assume that both $X$ and $Z$ experience a conditional pdf $p_{Y|X}$ and produce random variables $Y_1$ and $Y_2$ respectively, i.e., \begin{align} p_{Y_1}&=\sum_{x\in\mathcal{X}}p_{Y|X=x}p_X(x)\\ p_{Y_2}&=\sum_{x\in\mathcal{X}}p_{Y|Z=x}p_Z(z). \end{align} How one can find $\lambda$ such that the following inequality for KL divergrnce is satisfied \begin{align} \mathrm{D}(p_{Y|Z}||p_{Y_2})\leq \mathrm{D}(p_{Y|X}||p_{Y_1}). \end{align} Is it possible?

Let $X$ be a random variable which takes values in $\mathcal{X}$. Assume that we pass $X$ through two independent conditional pdf $p_{X_1|X}$ and $p_{X_1|X}$ and choose $X_1$ with probability $\lambda$ and $X_2$ with $\bar{\lambda}=1-\lambda$. Then, we have the following random variable $Z$: \begin{align} Z=\begin{cases} X_1&\text{with prob.}~ \lambda\\ X_2&\text{with prob.}~ \bar{\lambda}. \end{cases} \end{align} Therefore, the resulted probability density function of $Z$ is $\lambda p_{X_1}+\bar{\lambda}p_{X_2}$. Assume that $X_1,X_2$ also belong to $\mathcal{X}$. Now assume that both $X$ and $Z$ experience a conditional pdf $p_{Y|X}$ and produce random variables $Y_1$ and $Y_2$ respectively, i.e., \begin{align} p_{Y_1}&=\sum_{x\in\mathcal{X}}p_{Y|X=x}p_X(x)\\ p_{Y_2}&=\sum_{x\in\mathcal{X}}p_{Y|Z=x}p_Z(z). \end{align} How one can find $\lambda$ such that the following inequality for KL divergrnce is satisfied \begin{align} \mathrm{D}(p_{Y|Z}||p_{Y_2})\leq \mathrm{D}(p_{Y|X}||p_{Y_1}). \end{align} Is it possible?

Let $X$ be a random variable which takes values in $\mathcal{X}$. Assume that we pass $X$ through two independent conditional pdf $p_{X_1|X}$ and $p_{X_2|X}$ and choose $X_1$ with probability $\lambda$ and $X_2$ with $\bar{\lambda}=1-\lambda$. Then, we have the following random variable $Z$: \begin{align} Z=\begin{cases} X_1&\text{with prob.}~ \lambda\\ X_2&\text{with prob.}~ \bar{\lambda}. \end{cases} \end{align} Therefore, the resulted probability density function of $Z$ is $\lambda p_{X_1}+\bar{\lambda}p_{X_2}$. Assume that $X_1,X_2$ also belong to $\mathcal{X}$. Now assume that both $X$ and $Z$ experience a conditional pdf $p_{Y|X}$ and produce random variables $Y_1$ and $Y_2$ respectively, i.e., \begin{align} p_{Y_1}&=\sum_{x\in\mathcal{X}}p_{Y|X=x}p_X(x)\\ p_{Y_2}&=\sum_{x\in\mathcal{X}}p_{Y|Z=x}p_Z(z). \end{align} How one can find $\lambda$ such that the following inequality for KL divergrnce is satisfied \begin{align} \mathrm{D}(p_{Y|Z}||p_{Y_2})\leq \mathrm{D}(p_{Y|X}||p_{Y_1}). \end{align} Is it possible?

Notice added Draw attention by Math_Y
Bounty Started worth 50 reputation by Math_Y
added 221 characters in body; edited tags
Source Link
Math_Y
  • 287
  • 2
  • 16

Let $X\in\mathcal{X}$$X$ be a random variable which takes values in $\mathcal{X}$. Assume that we pass $X$ through two independent channelsconditional pdf $p_{X_1|X}$ and $p_{X_1|X}$ and choose $X_1$ with probability $\lambda$ and $X_2$ with $\bar{\lambda}$$\bar{\lambda}=1-\lambda$. Then, we have the following random variable $Z$: \begin{align} Z=\begin{cases} X_1&\text{with prob.}~ \lambda\\ X_2&\text{with prob.}~ \bar{\lambda}. \end{cases} \end{align} Therefore, the resulted probability density function of $Z$ is $\lambda p_{X_1}+\bar{\lambda}p_{X_2}$. Assume that $X_1,X_2$ also belong to $\mathcal{X}$. Now assume that both $X$ and $Z$ cross fromexperience a conditional pdf $p_{Y|X}$ and produce random variables $Y_1$ and $Y_2$ respectively, i. Howe., \begin{align} p_{Y_1}&=\sum_{x\in\mathcal{X}}p_{Y|X=x}p_X(x)\\ p_{Y_2}&=\sum_{x\in\mathcal{X}}p_{Y|Z=x}p_Z(z). \end{align} How one can find $\lambda$ such that the following inequality for KL divergrnce is satisfied \begin{align} \mathrm{D}(p_{Y|Z}||p_{Y_2})\leq \mathrm{D}(p_{Y|X}||p_{Y_1}). \end{align} Is it possible?

Let $X\in\mathcal{X}$ be a random variable. Assume that we pass $X$ through two independent channels $p_{X_1|X}$ and $p_{X_1|X}$ and choose $X_1$ with probability $\lambda$ and $X_2$ with $\bar{\lambda}$. \begin{align} Z=\begin{cases} X_1&\text{with prob.}~ \lambda\\ X_2&\text{with prob.}~ \bar{\lambda}. \end{cases} \end{align} Therefore, the resulted probability density function of $Z$ is $\lambda p_{X_1}+\bar{\lambda}p_{X_2}$. Assume that $X_1,X_2$ also belong to $\mathcal{X}$. Now assume that both $X$ and $Z$ cross from a conditional pdf $p_{Y|X}$ and produce random variables $Y_1$ and $Y_2$ respectively. How one can find $\lambda$ such that the following inequality for KL divergrnce is satisfied \begin{align} \mathrm{D}(p_{Y|Z}||p_{Y_2})\leq \mathrm{D}(p_{Y|X}||p_{Y_1}). \end{align} Is it possible?

Let $X$ be a random variable which takes values in $\mathcal{X}$. Assume that we pass $X$ through two independent conditional pdf $p_{X_1|X}$ and $p_{X_1|X}$ and choose $X_1$ with probability $\lambda$ and $X_2$ with $\bar{\lambda}=1-\lambda$. Then, we have the following random variable $Z$: \begin{align} Z=\begin{cases} X_1&\text{with prob.}~ \lambda\\ X_2&\text{with prob.}~ \bar{\lambda}. \end{cases} \end{align} Therefore, the resulted probability density function of $Z$ is $\lambda p_{X_1}+\bar{\lambda}p_{X_2}$. Assume that $X_1,X_2$ also belong to $\mathcal{X}$. Now assume that both $X$ and $Z$ experience a conditional pdf $p_{Y|X}$ and produce random variables $Y_1$ and $Y_2$ respectively, i.e., \begin{align} p_{Y_1}&=\sum_{x\in\mathcal{X}}p_{Y|X=x}p_X(x)\\ p_{Y_2}&=\sum_{x\in\mathcal{X}}p_{Y|Z=x}p_Z(z). \end{align} How one can find $\lambda$ such that the following inequality for KL divergrnce is satisfied \begin{align} \mathrm{D}(p_{Y|Z}||p_{Y_2})\leq \mathrm{D}(p_{Y|X}||p_{Y_1}). \end{align} Is it possible?

An inequlaityinequality of KL Divergence for two different distributiondistributions passing through a same channel

Source Link
Math_Y
  • 287
  • 2
  • 16
Loading