I have been working on a problem(alternate minimization) where I want to establish an inequality in which I am stuck.
An $\alpha$- parameterized version of the divergence(Kullback-Leibler) takes the following form:
$$I_{\alpha}(P,Q)=\frac{1}{\rho}\log\left[\frac{h(Q)^{1-\alpha}}{h(P)}\sum_{a\in A}P(a)Q(a)^{\alpha-1}\right] $$ where $\alpha=\frac{1}{1+\rho}, \rho>0$ and I let $h(P)=\left(\sum P(a)^{\alpha}\right)^{\frac{1}{\alpha}}$, $A$ is a finite set and $P, Q$ are distributions on $A$.
By Holder's inequality I have shown that $\sum_{a\in A}P(a)Q(a)^{\alpha-1}\ge h(Q)^{\alpha-1}h(P)$. So $I_{\alpha}(P,Q)\ge 0$ and is $0$ iff $P=Q$.
The problem is the following:
Let $P$ be a distribution and $E$ be a closed and convex set. Let $I_{\alpha}(P,Q^*)=\min_{Q\in E}I_{\alpha}(P,Q)$.
I want to establish $I_{\alpha}(P',P)+I_{\alpha}(P',Q')\ge I_{\alpha}(P',Q^*)$ for any $P'$ and any $Q'\in E$.
The equivalent form of the above ineq. is $$\frac{h(P)^{1-\alpha}}{h(P')}\sum_{a\in A}P'(a)P(a)^{\alpha-1}\cdot \frac{h(Q')^{1-\alpha}}{h(P')}\sum_{a\in A}P'(a)Q'(a)^{\alpha-1}\ge \frac{h(Q^*)^{1-\alpha}}{h(P')}\sum_{a\in A}P'(a)Q^*(a)^{\alpha-1}$$
Imre Csiszar has proved the same inequality for the Kullback Leibler divergence where he employs a derivative argument. When I do the similar thing I get the following:
Let $Q_t=(1-t)Q^*+tQ'\in E, 0\le t\le 1$. Then $$0\le \frac{1}{t}\left[I_{\alpha}(P,Q_t)-I_{\alpha}(P,Q^*)\right]=\left[\frac{d}{dt}I_{\alpha}(P,Q_t)\right]_{t=\tilde{t}}, \quad 0<\tilde{t}\le t$$
As $t\to 0$, I get the following: $$\frac{1}{h(Q^*)^{\alpha}}\sum_{a\in A}P(a)Q^*(a)^{\alpha-1}. \sum_{a\in A}Q'(a)Q^*(a)^{\alpha-1}\ge \sum_{a\in A}P(a)Q'(a)Q^*(a)^{\alpha-2}$$ I don't know how to use this to prove the desired inequality. Can anyone help? You may refer Csiszar's book on Information theory and statistics- a tutorial page no 445-446, available here http://www.nowpublishers.com/product.aspx?product=CIT&doi=0100000004
Update 14, September.
If I denote $\frac{P'(a)}{h(P')}$'s by $a_i$, $\frac{P(a)}{h(P)}$'s by $b_i$, $\frac{Q'(a)}{h(Q')}$'s by $c_i$, and $\frac{Q^{\star}(a)}{h(Q^{\star})}$'s by $d_i$, then a, b, c, d are vectors whose $\alpha$th norm is $1$. Problem would be solved if we show the following. $$\min_{\|a\|_{\alpha}=1}\sum_{i=1}^n a_i b_i^{\alpha-1}.\sum_{i=1}^n a_i c_i^{\alpha-1}-\sum_{i=1}^n a_i d_i^{\alpha-1}\ge \sum_{i=1}^n b_i d_i^{\alpha-1}. \sum_{i=1}^n c_i d_i^{\alpha-1}-\sum_{i=1}^n b_i c_i d_i^{\alpha-2}$$ as the right hand side is $\ge 0$.