total variation distance of product of measures - MathOverflow most recent 30 from http://mathoverflow.net 2013-05-23T12:12:36Z http://mathoverflow.net/feeds/question/79430 http://www.creativecommons.org/licenses/by-nc/2.5/rdf http://mathoverflow.net/questions/79430/total-variation-distance-of-product-of-measures total variation distance of product of measures Nima 2011-10-28T20:59:32Z 2011-10-29T05:27:03Z <p>Let $f, \hat{f}, g,$ and $\hat{g}$ be continuous probability densities. Define probability densities $p \propto fg$ and $\hat{p} \propto \hat{f}\hat{g}$. Is it true that \begin{align*} ||p - \hat{p}||_{1} \le || f - \hat{f}|| _{1} + ||g - \hat{g}||_1 \end{align*}</p> http://mathoverflow.net/questions/79430/total-variation-distance-of-product-of-measures/79445#79445 Answer by John Jiang for total variation distance of product of measures John Jiang 2011-10-29T02:45:59Z 2011-10-29T05:27:03Z <p>Here is a real counterexample, as verified by mathematica: so we take the same discrete two point space as below, and let $f=(u_1,u_2), \hat{f} = (v_1,v_2), g = (a_1,a_2), \hat{g}=(b_1,b_2)$, with the relation that $x_1 + x_2 = 1$ where $x \in {u,v,a,b}$. So your claim becomes in this special case</p> <p>$$|\frac{u_1 a_1}{u_1 p_1 + u_2 a_2} - \frac{v_1 b_1}{v_1 b_1 + v_2 b_2}| + | \frac{u_2 a_2}{u_1 a_1 + u_2 a_2} - \frac{v_2 b_2}{v_1 b_1 + v_2 b_2}|$$ $$\le |u_1 - v_1| + |u_2 - v_2| + |a_1 - b_1| + |a_2 - b_2|$$</p> <p>Now using the relation that if $x + y = 1$ and $w + z = 1$, then $|x - w| + |y - z| = 2 |x -w|$, we can simplify the above inequality to</p> <p>$$|\frac{u_1 a_1}{u_1 a_1 + u_2 a_2} - \frac{v_1 b_1}{v_1 b_1 + v_2 b_2}| \le |u_1 - v_1| + |a_1 - b_1|$$.</p> <p>What I then did is to subtract the square of the right hand side from the square of the left hand side, and call the resulting function $h(u,v,a,b)$, where $x := x_1$ for $x \in {u,v,a,b}$. Then I did some brute force search of maximum and found the following point:</p> <p>$$u=v =0.456239, a = 1/5, b = 1/2$$ If you plug it in to $h$ you should get $0.000652285 > 0$. So that means the left hand side is not always $\le$ the right hand side. Here are the mathematica code so that you can help me verify:</p> <p>q[u1_, u2_, a1_, a2_, v1_, v2_, b1_, b2_] := (u1 a1/(u1 a1 + u2 a2) - v1 b1/(v1 b1 + v2 b2))^2 - ( (u1 - v1)^2 + (a1 - b1)^2 + 2 Abs[(u1 - v1) (a1 - b1)]);</p> <p>h[u_, v_, a_, b_] := q[u, 1 - u, a, 1 - a, v, 1 - v, b, 1 - b];</p> <p>h[0.456239, 0.456239, 2/3, 2/5]</p> <p>= 0.000652285</p> <p>Edit: below was my earlier false counterexample. </p> <p>I believe it's false. Here is an argument. It suffices to consider discrete probability space, say the two point space $\Omega = { 0,1 }$. Now let $f =(u_1,1-u_1)$ be the probability mass function, so that means $f(0) = u_1$ and $f(1) = 1-u_1$. and similarly $\hat{f} = (u_2,1-u_2)$. Also I will let $g = \hat{g} = (p, 1-p)$. Then $\|g - \hat{g}\|_1 = 0$, so your inequality amounts to </p> <p>$|\frac{p u_1}{p u_1 + (1-p)(1-u_1)} - \frac{p u_2}{pu_2 + (1-p)(1-u_2)}| \le |u_1 - u_2|$.</p> <p>Now take $u_1 = 2/3$ and $u_2 = 1/3$, and let $1-p >> p$. Then you get a contradiction. </p>