Inequality involving probability measures - MathOverflow [closed] most recent 30 from http://mathoverflow.net 2013-05-21T08:24:13Z http://mathoverflow.net/feeds/question/32280 http://www.creativecommons.org/licenses/by-nc/2.5/rdf http://mathoverflow.net/questions/32280/inequality-involving-probability-measures Inequality involving probability measures Ashok 2010-07-17T11:56:31Z 2010-09-23T13:13:14Z <p>I have been working on a problem(alternate minimization) where I want to establish an inequality in which I am stuck.</p> <p>An $\alpha$- parameterized version of the divergence(Kullback-Leibler) takes the following form:</p> <p>$$I_{\alpha}(P,Q)=\frac{1}{\rho}\log\left[\frac{h(Q)^{1-\alpha}}{h(P)}\sum_{a\in A}P(a)Q(a)^{\alpha-1}\right]$$ where $\alpha=\frac{1}{1+\rho}, \rho>0$ and I let $h(P)=\left(\sum P(a)^{\alpha}\right)^{\frac{1}{\alpha}}$, $A$ is a finite set and $P, Q$ are distributions on $A$.</p> <p>By Holder's inequality I have shown that $\sum_{a\in A}P(a)Q(a)^{\alpha-1}\ge h(Q)^{\alpha-1}h(P)$. So $I_{\alpha}(P,Q)\ge 0$ and is $0$ iff $P=Q$.</p> <p>The problem is the following:</p> <p>Let $P$ be a distribution and $E$ be a closed and convex set. Let $I_{\alpha}(P,Q^*)=\min_{Q\in E}I_{\alpha}(P,Q)$.</p> <p>I want to establish $I_{\alpha}(P',P)+I_{\alpha}(P',Q')\ge I_{\alpha}(P',Q^*)$ for any $P'$ and any $Q'\in E$.</p> <p>The equivalent form of the above ineq. is $$\frac{h(P)^{1-\alpha}}{h(P')}\sum_{a\in A}P'(a)P(a)^{\alpha-1}\cdot \frac{h(Q')^{1-\alpha}}{h(P')}\sum_{a\in A}P'(a)Q'(a)^{\alpha-1}\ge \frac{h(Q^*)^{1-\alpha}}{h(P')}\sum_{a\in A}P'(a)Q^*(a)^{\alpha-1}$$</p> <p>Imre Csiszar has proved the same inequality for the Kullback Leibler divergence where he employs a derivative argument. When I do the similar thing I get the following:</p> <p>Let $Q_t=(1-t)Q^*+tQ'\in E, 0\le t\le 1$. Then $$0\le \frac{1}{t}\left[I_{\alpha}(P,Q_t)-I_{\alpha}(P,Q^*)\right]=\left[\frac{d}{dt}I_{\alpha}(P,Q_t)\right]_{t=\tilde{t}}, \quad 0&lt;\tilde{t}\le t$$</p> <p>As $t\to 0$, I get the following: $$\frac{1}{h(Q^*)^{\alpha}}\sum_{a\in A}P(a)Q^*(a)^{\alpha-1}. \sum_{a\in A}Q'(a)Q^*(a)^{\alpha-1}\ge \sum_{a\in A}P(a)Q'(a)Q^*(a)^{\alpha-2}$$ I don't know how to use this to prove the desired inequality. Can anyone help? You may refer Csiszar's book on Information theory and statistics- a tutorial page no 445-446, available here <a href="http://www.nowpublishers.com/product.aspx?product=CIT&amp;doi=0100000004" rel="nofollow">http://www.nowpublishers.com/product.aspx?product=CIT&amp;doi=0100000004</a></p> <p>Update 14, September.</p> <p>If I denote $\frac{P'(a)}{h(P')}$'s by $a_i$, $\frac{P(a)}{h(P)}$'s by $b_i$, $\frac{Q'(a)}{h(Q')}$'s by $c_i$, and $\frac{Q^{\star}(a)}{h(Q^{\star})}$'s by $d_i$, then <strong>a</strong>, <strong>b</strong>, <strong>c</strong>, <strong>d</strong> are vectors whose $\alpha$th norm is $1$. Problem would be solved if we show the following. $$\min_{\|a\|<em>{\alpha}=1}\sum</em>{i=1}^n a_i b_i^{\alpha-1}.\sum_{i=1}^n a_i c_i^{\alpha-1}-\sum_{i=1}^n a_i d_i^{\alpha-1}\ge \sum_{i=1}^n b_i d_i^{\alpha-1}. \sum_{i=1}^n c_i d_i^{\alpha-1}-\sum_{i=1}^n b_i c_i d_i^{\alpha-2}$$ as the right hand side is $\ge 0$.</p> http://mathoverflow.net/questions/32280/inequality-involving-probability-measures/33379#33379 Answer by robin girard for Inequality involving probability measures robin girard 2010-07-26T10:38:07Z 2010-07-26T11:01:07Z <p>If the inequality is true, you will find a proof with your technic by changing the transport you use: $Q_t$ . This transport is "adapted" to the geometry induced by KL but not by your $\alpha$-KL. I'll try to find the time to find the good transformation from my side and add it to the answer, or maibe you'll find it yourself :) </p> <hr> <p><strong>Remarks</strong></p> <p>You define the $\alpha$- parameterized version of the divergence(Kullback-Leibler), can you tell us why you call it like that (is the kullback leibler divergence obtained when $p$ goes to infinity) ? You seem to be sure that your inquality is true, you may have a reference or is it intuition ?</p> <p>These optimal transformations things makes me remember about Theorem 5.24 and Example 5.27 of <a href="http://www.umpa.ens-lyon.fr/~cvillani/Cedrif/B07D.StFlour.pdf" rel="nofollow">http://www.umpa.ens-lyon.fr/~cvillani/Cedrif/B07D.StFlour.pdf</a> maybe this very general theorem can also help you ? </p> http://mathoverflow.net/questions/32280/inequality-involving-probability-measures/38697#38697 Answer by S. Sra for Inequality involving probability measures S. Sra 2010-09-14T15:35:43Z 2010-09-14T15:35:43Z <p>If you can show that your $\alpha$-divergence above is a Bregman divergence that is convex in both its arguments, then the following paper points out a result that implies the abovementioned <em>four-points property</em> that you are trying to prove.</p> <p>Link: <a href="http://math.haifa.ac.il/yair/AORfulltext.pdf" rel="nofollow">http://math.haifa.ac.il/yair/AORfulltext.pdf</a></p>