| |
|
Post Closed as "no longer relevant" by Scott Morrison♦
|
occurred Nov 15 2010 at 23:18
|
|
|
|
|
| |
|
Post Undeleted by Pete L. Clark, Mariano Suárez-Alvarez, Scott Morrison♦
|
occurred Nov 6 2010 at 10:16
|
|
|
|
|
| |
|
Post Deleted by Ashok
|
occurred Oct 20 2010 at 7:13
|
|
|
|
|
|
9
|
|
edited Sep 23 2010 at 13:13
|
I have been working on a problem(alternate minimization) where I want to establish an inequality in which I am stuck.
An $\alpha$- parameterized version of the divergence(Kullback-Leibler) takes the following form:
$$I_{\alpha}(P,Q)=\frac{1}{\rho}\log\left[\frac{h(Q)^{1-\alpha}}{h(P)}\sum_{a\in A}P(a)Q(a)^{\alpha-1}\right] $$
where $\alpha=\frac{1}{1+\rho}, \rho>0$
and I let $h(P)=\left(\sum P(a)^{\alpha}\right)^{\frac{1}{\alpha}}$, $A$ is a finite set and $P, Q$ are distributions on $A$.
By Holder's inequality I have shown that $\sum_{a\in A}P(a)Q(a)^{\alpha-1}\ge h(Q)^{\alpha-1}h(P)$. So $I_{\alpha}(P,Q)\ge 0$ and is $0$ iff $P=Q$.
The problem is the following:
Let $P$ be a distribution and $E$ be a closed and convex set. Let $I_{\alpha}(P,Q^*)=\min_{Q\in E}I_{\alpha}(P,Q)$.
I want to establish $I_{\alpha}(P',P)+I_{\alpha}(P',Q')\ge I_{\alpha}(P',Q^*)$ for any $P'$ and any $Q'\in E$.
The equivalent form of the above ineq. is
$$\frac{h(P)^{1-\alpha}}{h(P')}\sum_{a\in A}P'(a)P(a)^{\alpha-1}\cdot \frac{h(Q')^{1-\alpha}}{h(P')}\sum_{a\in A}P'(a)Q'(a)^{\alpha-1}\ge \frac{h(Q^*)^{1-\alpha}}{h(P')}\sum_{a\in A}P'(a)Q^*(a)^{\alpha-1}$$
Imre Csiszar has proved the same inequality for the Kullback Leibler divergence where he employs a derivative argument. When I do the similar thing I get the following:
Let $Q_t=(1-t)Q^*+tQ'\in E, 0\le t\le 1$. Then
$$0\le \frac{1}{t}\left[I_{\alpha}(P,Q_t)-I_{\alpha}(P,Q^*)\right]=\left[\frac{d}{dt}I_{\alpha}(P,Q_t)\right]_{t=\tilde{t}}, \quad 0<\tilde{t}\le t$$
As $t\to 0$, I get the following:
$$\frac{1}{h(Q^*)^{\alpha}}\sum_{a\in A}P(a)Q^*(a)^{\alpha-1}. \sum_{a\in A}Q'(a)Q^*(a)^{\alpha-1}\ge \sum_{a\in A}P(a)Q'(a)Q^*(a)^{\alpha-2}$$
I don't know how to use this to prove the desired inequality. Can anyone help?
You may refer Csiszar's book on Information theory and statistics- a tutorial page no 445-446, available here http://www.nowpublishers.com/product.aspx?product=CIT&doi=0100000004
Update 14, September.
If I denote $\frac{P'(a)}{h(P')}$'s by $a_i$, $\frac{P(a)}{h(P)}$'s by $b_i$, $\frac{Q'(a)}{h(Q')}$'s by $c_i$, and $\frac{Q^{\star}(a)}{h(Q^{\star})}$'s by $d_i$, then a, b, c, d are vectors whose $\alpha$th norm is $1$.
Problem would be solved if we show the following.
$$\min_{\|a\|{\alpha}=1}\sum{i=1}^n a_i b_^{\alpha-1}.\sum_{i=1}^n b_i^{\alpha-1}.\sum_{i=1}^n a_i c_i^{\alpha-1}-\sum_{i=1}^n a_i d_^{\alpha-1}\ge d_i^{\alpha-1}\ge \sum_{i=1}^n b_i d_^{\alpha-1}d_i^{\alpha-1}. \sum_{i=1}^n c_i d_i^{\alpha-1}-\sum_{i=1}^n b_i c_i d_i^{\alpha-2}$$ as the right hand side is $\ge 0$.
|
|
|
|
8
|
|
edited Sep 15 2010 at 9:18
|
I have been working on a problem(alternate minimization) where I want to establish an inequality in which I am stuck.
An $\alpha$- parameterized version of the divergence(Kullback-Leibler) takes the following form:
$$I_{\alpha}(P,Q)=\frac{1}{\rho}\log\left[\frac{h(Q)^{1-\alpha}}{h(P)}\sum_{a\in A}P(a)Q(a)^{\alpha-1}\right] $$
where $\alpha=\frac{1}{1+\rho}, \rho>0$
and I let $h(P)=\left(\sum P(a)^{\alpha}\right)^{\frac{1}{\alpha}}$, $A$ is a finite set and $P, Q$ are distributions on $A$.
By Holder's inequality I have shown that $\sum_{a\in A}P(a)Q(a)^{\alpha-1}\ge h(Q)^{\alpha-1}h(P)$. So $I_{\alpha}(P,Q)\ge 0$ and is $0$ iff $P=Q$.
The problem is the following:
Let $P$ be a distribution and $E$ be a closed and convex set. Let $I_{\alpha}(P,Q^*)=\min_{Q\in E}I_{\alpha}(P,Q)$.
I want to establish $I_{\alpha}(P',P)+I_{\alpha}(P',Q')\ge I_{\alpha}(P',Q^*)$ for any $P'$ and any $Q'\in E$.
The equivalent form of the above ineq. is
$$\frac{h(P)^{1-\alpha}}{h(P')}\sum_{a\in A}P'(a)P(a)^{\alpha-1}\cdot \frac{h(Q')^{1-\alpha}}{h(P')}\sum_{a\in A}P'(a)Q'(a)^{\alpha-1}\ge \frac{h(Q^*)^{1-\alpha}}{h(P')}\sum_{a\in A}P'(a)Q^*(a)^{\alpha-1}$$
Imre Csiszar has proved the same inequality for the Kullback Leibler divergence where he employs a derivative argument. When I do the similar thing I get the following:
Let $Q_t=(1-t)Q^*+tQ'\in E, 0\le t\le 1$. Then
$$0\le \frac{1}{t}\left[I_{\alpha}(P,Q_t)-I_{\alpha}(P,Q^*)\right]=\left[\frac{d}{dt}I_{\alpha}(P,Q_t)\right]_{t=\tilde{t}}, \quad 0<\tilde{t}\le t$$
As $t\to 0$, I get the following:
$$\frac{1}{h(Q^*)^{\alpha}}\sum_{a\in A}P(a)Q^*(a)^{\alpha-1}. \sum_{a\in A}Q'(a)Q^*(a)^{\alpha-1}\ge \sum_{a\in A}P(a)Q'(a)Q^*(a)^{\alpha-2}$$
I don't know how to use this to prove the desired inequality. Can anyone help?
You may refer Csiszar's book on Information theory and statistics- a tutorial page no 445-446, available here http://www.nowpublishers.com/product.aspx?product=CIT&doi=0100000004
Update 14, September.
If I denote $\frac{P'(a)}{h(P')}$'s by $a_i$, $\frac{P(a)}{h(P)}$'s by $b_i$, $\frac{Q'(a)}{h(Q')}$'s by $c_i$, and $\frac{Q^{\star}(a)}{h(Q^{\star})}$'s by $d_i$, then a, b, c, d are vectors whose $\alpha$th norm is $1$.
Problem would be solved if we show the following.
$\min_{\|a\|$\min_{\|a\|{\alpha}=1}\sum{i=1}^n a_i b_^{\alpha-1}.\sum_{i=1}^n a_i c_i^{\alpha-1}-\sum_{i=1}^n a_i d_^{\alpha-1}\ge \sum_{i=1}^n b_i d_^{\alpha-1}. \sum_{i=1}^n c_i d_i^{\alpha-1}-\sum_{i=1}^n b_i c_i d_i^{\alpha-2}$ d_i^{\alpha-2}$$ as the right hand side is $\ge 0$.
|
|
|
|
7
|
|
edited Sep 15 2010 at 9:12
|
I have been working on a problem(alternate minimization) where I want to establish an inequality in which I am stuck.
An $\alpha$- parameterized version of the divergence(Kullback-Leibler) takes the following form:
$$I_{\alpha}(P,Q)=\frac{1}{\rho}\log\left[\frac{h(Q)^{1-\alpha}}{h(P)}\sum_{a\in A}P(a)Q(a)^{\alpha-1}\right] $$
where $\alpha=\frac{1}{1+\rho}, \rho>0$
and I let $h(P)=\left(\sum P(a)^{\alpha}\right)^{\frac{1}{\alpha}}$, $A$ is a finite set and $P, Q$ are distributions on $A$.
By Holder's inequality I have shown that $\sum_{a\in A}P(a)Q(a)^{\alpha-1}\ge h(Q)^{\alpha-1}h(P)$. So $I_{\alpha}(P,Q)\ge 0$ and is $0$ iff $P=Q$.
The problem is the following:
Let $P$ be a distribution and $E$ be a closed and convex set. Let $I_{\alpha}(P,Q^*)=\min_{Q\in E}I_{\alpha}(P,Q)$.
I want to establish $I_{\alpha}(P',P)+I_{\alpha}(P',Q')\ge I_{\alpha}(P',Q^*)$ for any $P'$ and any $Q'\in E$.
The equivalent form of the above ineq. is
$$\frac{h(P)^{1-\alpha}}{h(P')}\sum_{a\in A}P'(a)P(a)^{\alpha-1}\cdot \frac{h(Q')^{1-\alpha}}{h(P')}\sum_{a\in A}P'(a)Q'(a)^{\alpha-1}\ge \frac{h(Q^*)^{1-\alpha}}{h(P')}\sum_{a\in A}P'(a)Q^*(a)^{\alpha-1}$$
Imre Csiszar has proved the same inequality for the Kullback Leibler divergence where he employs a derivative argument. When I do the similar thing I get the following:
Let $Q_t=(1-t)Q^*+tQ'\in E, 0\le t\le 1$. Then
$$0\le \frac{1}{t}\left[I_{\alpha}(P,Q_t)-I_{\alpha}(P,Q^*)\right]=\left[\frac{d}{dt}I_{\alpha}(P,Q_t)\right]_{t=\tilde{t}}, \quad 0<\tilde{t}\le t$$
As $t\to 0$, I get the following:
$$\frac{1}{h(Q^*)^{\alpha}}\sum_{a\in A}P(a)Q^*(a)^{\alpha-1}. \sum_{a\in A}Q'(a)Q^*(a)^{\alpha-1}\ge \sum_{a\in A}P(a)Q'(a)Q^*(a)^{\alpha-2}$$
I don't know how to use this to prove the desired inequality. Can anyone help?
You may refer Csiszar's book on Information theory and statistics- a tutorial page no 445-446, available here http://www.nowpublishers.com/product.aspx?product=CIT&doi=0100000004
Update 14, September.
If I denote $\frac{P'(a)}{h(P')}$'s by $a_i$, $\frac{P(a)}{h(P)}$'s by $b_i$, $\frac{Q'(a)}{h(Q')}$'s by $c_i$, and $\frac{Q^(a)}{h(Q^)}$ 's \frac{Q^{\star}(a)}{h(Q^{\star})}$'s by $d_i$, then a, b, c, d are vectors whose $\alpha$th norm is $1$.
Problem would be solved if we show the following.
$$\min_{\|a\|\min_{\|a\|{\alpha}=1}\sum{i=1}^n a_i b_^{\alpha-1}.\sum_{i=1}^n a_i c_i^{\alpha-1}-\sum_{i=1}^n a_i d_^{\alpha-1}\ge \sum_{i=1}^n b_i d_^{\alpha-1}. \sum_{i=1}^n c_i d_i^{\alpha-1}-\sum_{i=1}^n b_i c_i d_i^{\alpha-2}$$ d_i^{\alpha-2}$ as the right hand side is $\ge 0$.
|
|
|
|
6
|
|
edited Sep 14 2010 at 13:33
|
I have been working on a problem(alternate minimization) where I want to establish an inequality in which I am stuck.
An $\alpha$- parameterized version of the divergence(Kullback-Leibler) takes the following form:
$$I_{\alpha}(P,Q)=\frac{1}{\rho}\log\left[\frac{h(Q)^{1-\alpha}}{h(P)}\sum_{a\in A}P(a)Q(a)^{\alpha-1}\right] $$
where $\alpha=\frac{1}{1+\rho}, \rho>0$
and I let $h(P)=\left(\sum P(a)^{\alpha}\right)^{\frac{1}{\alpha}}$, $A$ is a finite set and $P, Q$ are distributions on $A$.
By Holder's inequality I have shown that $\sum_{a\in A}P(a)Q(a)^{\alpha-1}\ge h(Q)^{\alpha-1}h(P)$. So $I_{\alpha}(P,Q)\ge 0$ and is $0$ iff $P=Q$.
The problem is the following:
Let $P$ be a distribution and $E$ be a closed and convex set. Let $I_{\alpha}(P,Q^*)=\min_{Q\in E}I_{\alpha}(P,Q)$.
I want to establish $I_{\alpha}(P',P)+I_{\alpha}(P',Q')\ge I_{\alpha}(P',Q^*)$ for any $P'$ and any $Q'\in E$.
The equivalent form of the above ineq. is
$$\frac{h(P)^{1-\alpha}}{h(P')}\sum_{a\in A}P'(a)P(a)^{\alpha-1}\cdot \frac{h(Q')^{1-\alpha}}{h(P')}\sum_{a\in A}P'(a)Q'(a)^{\alpha-1}\ge \frac{h(Q^*)^{1-\alpha}}{h(P')}\sum_{a\in A}P'(a)Q^*(a)^{\alpha-1}$$
Imre Csiszar has proved the same inequality for the Kullback Leibler divergence where he employs a derivative argument. When I do the similar thing I get the following:
Let $Q_t=(1-t)Q^*+tQ'\in E, 0\le t\le 1$. Then
$$0\le \frac{1}{t}\left[I_{\alpha}(P,Q_t)-I_{\alpha}(P,Q^*)\right]=\left[\frac{d}{dt}I_{\alpha}(P,Q_t)\right]_{t=\tilde{t}}, \quad 0<\tilde{t}\le t$$
As $t\to 0$, I get the following:
$$\frac{1}{h(Q^*)^{\alpha}}\sum_{a\in A}P(a)Q^*(a)^{\alpha-1}. \sum_{a\in A}Q'(a)Q^*(a)^{\alpha-1}\ge \sum_{a\in A}P(a)Q'(a)Q^*(a)^{\alpha-2}$$
I don't know how to use this to prove the desired inequality. Can anyone help?
You may refer Csiszar's book on Information theory and statistics- a tutorial page no 445-446, available here http://www.nowpublishers.com/product.aspx?product=CIT&doi=0100000004
Update 14, September.
If I denote $\frac{P'(a)}{h(P')}$'s by $a_i$, $\frac{P(a)}{h(P)}$'s by $b_i$, $\frac{Q'(a)}{h(Q')}$'s by $c_i$, and $\frac{Q^(a)}{h(Q^)}$'s }$ 's by $d_i$, then a, b, c, d are vectors whose $\alpha$th norm is $1$.
Problem would be solved if we show the following.
$$\min_{\|a\|{\alpha}=1}\sum{i=1}^n a_i b_^{\alpha-1}.\sum_{i=1}^n a_i c_i^{\alpha-1}-\sum_{i=1}^n a_i d_^{\alpha-1}\ge \sum_{i=1}^n b_i d_^{\alpha-1}. \sum_{i=1}^n c_i d_i^{\alpha-1}-\sum_{i=1}^n b_i c_i d_i^{\alpha-2}$$ as the right hand side is $\ge 0$.
|
|
|
|
5
|
|
edited Sep 14 2010 at 9:37
|
I have been working on a problem(alternate minimization) where I want to establish an inequality in which I am stuck.
An $\alpha$- parameterized version of the divergence(Kullback-Leibler) takes the following form:
$$I_{\alpha}(P,Q)=\frac{1}{\rho}\log\left[\frac{h(Q)^{1-\alpha}}{h(P)}\sum_{a\in A}P(a)Q(a)^{\alpha-1}\right] $$
where $\alpha=\frac{1}{1+\rho}, \rho>0$
and I let $h(P)=\left(\sum P(a)^{\alpha}\right)^{\frac{1}{\alpha}}$, $A$ is a finite set and $P, Q$ are distributions on $A$.
By Holder's inequality I have shown that $\sum_{a\in A}P(a)Q(a)^{\alpha-1}\ge h(Q)^{\alpha-1}h(P)$. So $I_{\alpha}(P,Q)\ge 0$ and is $0$ iff $P=Q$.
The problem is the following:
Let $P$ be a distribution and $E$ be a closed and convex set. Let $I_{\alpha}(P,Q^*)=\min_{Q\in E}I_{\alpha}(P,Q)$.
I want to establish $I_{\alpha}(P',P)+I_{\alpha}(P',Q')\ge I_{\alpha}(P',Q^*)$ for any $P'$ and any $Q'\in E$.
The equivalent form of the above ineq. is
$$\frac{h(P)^{1-\alpha}}{h(P')}\sum_{a\in A}P'(a)P(a)^{\alpha-1}\cdot \frac{h(Q')^{1-\alpha}}{h(P')}\sum_{a\in A}P'(a)Q'(a)^{\alpha-1}\ge \frac{h(Q^*)^{1-\alpha}}{h(P')}\sum_{a\in A}P'(a)Q^*(a)^{\alpha-1}$$
Imre Csiszar has proved the same inequality for the Kullback Leibler divergence where he employs a derivative argument. When I do the similar thing I get the following:
Let $Q_t=(1-t)Q^*+tQ'\in E, 0\le t\le 1$. Then
$$0\le \frac{1}{t}\left[I_{\alpha}(P,Q_t)-I_{\alpha}(P,Q^*)\right]=\left[\frac{d}{dt}I_{\alpha}(P,Q_t)\right]_{t=\tilde{t}}, \quad 0<\tilde{t}\le t$$
As $t\to 0$, I get the following:
$$\frac{1}{h(Q^*)^{\alpha}}\sum_{a\in A}P(a)Q^*(a)^{\alpha-1}. \sum_{a\in A}Q'(a)Q^*(a)^{\alpha-1}\ge \sum_{a\in A}P(a)Q'(a)Q^*(a)^{\alpha-2}$$
I don't know how to use this to prove the desired inequality. Can anyone help?
You may refer Csiszar's book on Information theory and statistics- a tutorial page no 445-446, available here http://www.nowpublishers.com/product.aspx?product=CIT&doi=0100000004
Update 14, September.
If I denote $\frac{P'(a)}{h(P')}$'s by $a_i$, $\frac{P(a)}{h(P)}$'s by $b_i$, $\frac{Q'(a)}{h(Q')}$'s by $c_i$, and $\frac{Q^(a)}{h(Q^)}$'s by $d_i$, then a, b, c, d are vectors whose $\alpha$th norm is $1$.
Problem would be solved if we show the following.
$$\min_{\|a\|{\alpha}=1}\sum{i=1}^n a_i b_^{\alpha-1}.\sum_{i=1}^n a_i c_i^{\alpha-1}-\sum_{i=1}^n a_i d_^{\alpha-1}\ge \sum_{i=1}^n b_i d_^{\alpha-1}. \sum_{i=1}^n c_i d_i^{\alpha-1}-\sum_{i=1}^n b_i c_i d_i^{\alpha-2}$$ as the right hand side is $\ge 0$.
|
|
|
|
4
|
|
edited Sep 14 2010 at 8:42
|
I have been working on a problem(alternate minimization) where I want to establish an inequality in which I am stuck.
An $\alpha$- parameterized version of the divergence(Kullback-Leibler) takes the following form:
$$I_{\alpha}(P,Q)=\frac{1}{\rho}\log\left[\frac{h(Q)^{1-\alpha}}{h(P)}\sum_{a\in A}P(a)Q(a)^{\alpha-1}\right] $$
where $\alpha=\frac{1}{1+\rho}, \rho>0$
and I let $h(P)=\left(\sum P(a)^{\alpha}\right)^{\frac{1}{\alpha}}$, $A$ is a finite set and $P, Q$ are distributions on $A$.
By Holder's inequality I have shown that $\sum_{a\in A}P(a)Q(a)^{\alpha-1}\ge h(Q)^{\alpha-1}h(P)$. So $I_{\alpha}(P,Q)\ge 0$ and is $0$ iff $P=Q$.
The problem is the following:
Let $P$ be a distribution and $E$ be a closed and convex set. Let $I_{\alpha}(P,Q^*)=\min_{Q\in E}I_{\alpha}(P,Q)$.
I want to establish $I_{\alpha}(P',P)+I_{\alpha}(P',Q')\ge I_{\alpha}(P',Q^*)$ for any $P'$ and any $Q'\in E$.
The equivalent form of the above ineq. is
$$\frac{h(P)^{1-\alpha}}{h(P')}\sum_{a\in A}P'(a)P(a)^{\alpha-1}\cdot \frac{h(Q')^{1-\alpha}}{h(P')}\sum_{a\in A}P'(a)Q'(a)^{\alpha-1}\ge \frac{h(Q^*)^{1-\alpha}}{h(P')}\sum_{a\in A}P'(a)Q^*(a)^{\alpha-1}$$
Imre Csiszar has proved the same inequality for the Kullback Leibler divergence where he employs a derivative argument. When I do the similar thing I get the following:
Let $Q_t=(1-t)Q^*+tQ'\in E, 0\le t\le 1$. Then
$$0\le \frac{1}{t}\left[I_{\alpha}(P,Q_t)-I_{\alpha}(P,Q^*)\right]=\left[\frac{d}{dt}I_{\alpha}(P,Q_t)\right]_{t=\tilde{t}}, \quad 0<\tilde{t}\le t$$
As $t\to 0$, I get the following:
$$\frac{1}{h(Q^*)^{\alpha}}\sum_{a\in A}P(a)Q^*(a)^{\alpha-1}\ge A}P(a)Q^*(a)^{\alpha-1}. \sum_{a\in A}Q'(a)Q^*(a)^{\alpha-1}\ge \sum_{a\in A}P(a)Q'(a)Q^*(a)^{\alpha-2}$$
I don't know how to use this to prove the desired inequality. Can anyone help?
You may refer Csiszar's book on Information theory and statistics- a tutorial page no 445-446, available here http://www.nowpublishers.com/product.aspx?product=CIT&doi=0100000004
|
|
|
|
3
|
|
edited Jul 18 2010 at 1:07
|
I have been working on a problem(alternate minimization) where I want to establish an inequality in which I am stuck.
An $\alpha$- parameterized version of the divergence(Kullback-Leibler) takes the following form:
$I_{\alpha}(P,Q)=\displaystyle\frac{1}{\rho}\log\left[\frac{h(Q)^{1-\alpha}}{h(P)}\sum_{a\in $I_{\alpha}(P,Q)=\frac{1}{\rho}\log\left[\frac{h(Q)^{1-\alpha}}{h(P)}\sum_{a\in A}P(a)Q(a)^{\alpha-1}\right] $$
where $\alpha=\frac{1}{1+\rho}, \rho>0$
and I let $h(P)=\left(\sum P(a)^{\alpha}\right)^{\frac{1}{\alpha}}$, $A$ is a finite set and $P, Q$ are distributions on $A$.
By Holder's inequality I have shown that $\sum_{a\in A}P(a)Q(a)^{\alpha-1}\ge h(Q)^{\alpha-1}h(P)$. So $I_{\alpha}(P,Q)\ge 0$ and is $0$ iff $P=Q$.
The problem is the following:
Let $P$ be a distribution and $E$ be a closed and convex set. Let $I_{\alpha}(P,Q^*)=\min_{Q\in E}I_{\alpha}(P,Q)$.
I want to establish $I_{\alpha}(P',P)+I_{\alpha}(P',Q')\ge I_{\alpha}(P',Q^*)$ for any $P'$ and any $Q'\in E$.
The equivalent form of the above ineq. is
$\frac{h(P)^{1-\alpha}}{h(P')}\sum_{a\in $\frac{h(P)^{1-\alpha}}{h(P')}\sum_{a\in A}P'(a)P(a)^{\alpha-1}\cdot \frac{h(Q')^{1-\alpha}}{h(P')}\sum_{a\in A}P'(a)Q'(a)^{\alpha-1}\ge \frac{h(Q^*)^{1-\alpha}}{h(P')}\sum_{a\in A}P'(a)Q^*(a)^{\alpha-1}$A}P'(a)Q^*(a)^{\alpha-1}$$
Imre Csiszar has proved the same inequality for the Kullback Leibler divergence where he employs a derivative argument. When I do the similar thing I get the following:
Let $Q_t=(1-t)Q^*+tQ'\in E, 0\le t\le 1$. Then
$0\le $0\le \frac{1}{t}\left[I_{\alpha}(P,Q_t)-I_{\alpha}(P,Q^*)\right]=\left[\frac{d}{dt}I_{\alpha}(P,Q_t)\right]_{t=\tilde{t}}, \quad 0<\tilde{t}\le t$t$$
As $t\to 0$, I get the following:
$\frac{1}{h(Q^*)^{\alpha}}\sum_{a\in $\frac{1}{h(Q^*)^{\alpha}}\sum_{a\in A}P(a)Q^*(a)^{\alpha-1}\ge \sum_{a\in A}P(a)Q'(a)Q^*(a)^{\alpha-2}$
A}P(a)Q'(a)Q^*(a)^{\alpha-2}$$
I don't know how to use this to prove the desired inequality. Can anyone help?
You may refer Csiszar's book on Information theory and statistics- a tutorial page no 445-446, available here http://www.nowpublishers.com/product.aspx?product=CIT&doi=0100000004
|
|
|
|
2
|
|
edited Jul 17 2010 at 12:09
|
|
|
|
|
|
1
|
|
asked Jul 17 2010 at 11:56
|
Inequality involving probability measures
I have been working on a problem(alternate minimization) where I want to establish an inequality in which I am stuck.
An $\alpha$- parameterized version of the divergence(Kullback-Leibler) takes the following form:
$I_{\alpha}(P,Q)=\displaystyle\frac{1}{\rho}\log\left[\frac{h(Q)^{1-\alpha}}{h(P)}\sum_{a\in A}P(a)Q(a)^{\alpha-1}\right] $ where $\alpha=\frac{1}{1+\rho}, \rho>0$
and I let $h(P)=\left(\sum P(a)^{\alpha}\right)^{\frac{1}{\alpha}}$, $A$ is a finite set and $P, Q$ are distributions on $A$.
By Holder's inequality I have shown that $\sum_{a\in A}P(a)Q(a)^{\alpha-1}\ge h(Q)^{\alpha-1}h(P)$. So $I_{\alpha}(P,Q)\ge 0$ and is $0$ iff $P=Q$.
The problem is the following:
Let $P$ be a distribution and $E$ be a closed and convex set. Let $I_{\alpha}(P,Q^*)=\min_{Q\in E}I_{\alpha}(P,Q)$.
I want to establish $I_{\alpha}(P',P)+I_{\alpha}(P',Q')\ge I_{\alpha}(P',Q^*)$ for any $P'$ and any $Q'\in E$.
The equivalent form of the above ineq. is
$\frac{h(P)^{1-\alpha}}{h(P')}\sum_{a\in A}P'(a)P(a)^{\alpha-1}\cdot \frac{h(Q')^{1-\alpha}}{h(P')}\sum_{a\in A}P'(a)Q'(a)^{\alpha-1}\ge \frac{h(Q^*)^{1-\alpha}}{h(P')}\sum_{a\in A}P'(a)Q^*(a)^{\alpha-1}$
Imre Csiszar has proved the same inequality for the Kullback Leibler divergence where he employs a derivative argument. When I do the similar thing I get the following:
Let $Q_t=(1-t)Q^*+tQ'\in E, 0\le t\le 1$. Then
$0\le \frac{1}{t}\left[I_{\alpha}(P,Q_t)-I_{\alpha}(P,Q^*)\right]=\left[\frac{d}{dt}I_{\alpha}(P,Q_t)\right]_{t=\tilde{t}}, 0<\tilde{t}\le t$
As $t\to 0$, I get the following:
$\frac{1}{h(Q^*)^{\alpha}}\sum_{a\in A}P(a)Q^*(a)^{\alpha-1}\ge \sum_{a\in A}P(a)Q'(a)Q^*(a)^{\alpha-2}$
I don't know how to use this to prove the desired inequality. Can anyone help?
You may refer Csiszar's book on Information theory and statistics- a tutorial page no 445-446, available here http://www.nowpublishers.com/product.aspx?product=CIT&doi=0100000004
|
|
|