Timeline for Lower semicontinuity of Kullback-Leibler divergence
Current License: CC BY-SA 3.0
6 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Dec 30, 2011 at 13:55 | vote | accept | Michele | ||
| Dec 30, 2011 at 13:55 | comment | added | Michele | Ashok, I found the paper, man thanks. Michele | |
| Dec 30, 2011 at 13:45 | comment | added | Ashok | For example, you can find a proof in the paper (see section III) titled "Random coding strategies for minimum entropy" published in IT Transactions. | |
| Dec 30, 2011 at 13:27 | comment | added | Ashok | Yes Michele, you are right. My proof would not work in the case you have stated. But I am sure that its lower semi continuous. | |
| Dec 30, 2011 at 12:49 | comment | added | Michele | Surely it is not always the case that if $(P_n,Q_n)\rightarrow (P,Q)$ then $\lim_{n\rightarrow \infty} D(P_n||Q_n)= D(P||Q)$, as you say. If $Q$ is the frontier of the PMF simplex (that is, $Q(x)=0$ for some $x$), consider a sequence $(P_n,Q_n)$ with $Q_n=Q$ and $P_n\rightarrow P$, where $P(x)=0$ whenever $Q(x)=0$. Moreover, assume all the $P_n$'s are in the interior of the simplex, that is they are all strictly positive on every $x$. Then, for each $n$, $D(P_n||Q_n)=+\infty$, hence $\lim_{n\rightarrow \infty} D(P_n||Q_n)=+\infty$, whereas $D(P||Q)<+\infty$. | |
| Dec 30, 2011 at 8:40 | history | answered | Ashok | CC BY-SA 3.0 |