Suppose two dependent random variables $X$ and $V$ from finite alphabets $\mathcal{V}$ and $\mathcal{X}$ with known joint and marginal distributions are given. Let $P_{XV}$ and $P_X$ and $P_V$ are the joint and marginal distributions.

We want to generate another random variable $Z$ (on a finite alphabet $\mathcal{Z}$) from $X$ (hence $V-X-Z$ forms a Markov chain) such that for a fixed $\epsilon>0$ we have the following total variation distance constraint $$||P_{VZ}-P_VP_Z||_{TV}\leq \epsilon.$$

My question is what is the maximum possible value for $$||P_{XZ}-P_XP_Z||_{TV}.$$ The maximum here is taken over all conditional distribution $P_{Z|X}$ which satisfies the constraint $||P_{VZ}-P_VP_Z||_{TV}\leq \epsilon.$

Note that it can be easily shown that if both $V$ and $X$ are dependent *binary* random variables, then if $||P_{VZ}-P_VP_Z||_{TV}=0$ implies $||P_{XZ}-P_XP_Z||_{TV}=0$, in other words, independence of $V$ and $Z$ in Markov chain $V-X-Z$ implies independence of $X$ and $Z$ too.

I should mention that my original problem was bounding the relative entropy of $D(P_{XZ}||P_XP_Z)$ subject to $D(P_{VZ}||P_VP_Z)\leq \epsilon$, however I think the above problem can give some insight as to how attack relative entropy problem.

Any comment is appreciated.

3more comments