Timeline for Can KL divergence go to 0, but $E[\log(p/q)^2]$ diverge in certain cases?
Current License: CC BY-SA 4.0
8 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Nov 21 at 15:03 | history | edited | gmvh | CC BY-SA 4.0 |
MathJax'ed title, this bumps the post, but it doesn't have an answer and is eligible for a bump by the Community bot anyway
|
| Oct 13, 2010 at 21:56 | comment | added | rasper | Maybe my wording is not clear. The quantifiers are: Exists P such that For All eps,C,A Exists q_eps such that KL < eps and second moment >= C*eps^A | |
| Oct 13, 2010 at 21:54 | comment | added | rasper | Thanks. That's an interesting case, but does not follow the requirements: $p$ needs to be fixed. It is $q$ that can change depending on $\epsilon$. | |
| Oct 13, 2010 at 17:12 | comment | added | Paul Yuryev | Second moment does not necessarily exist in the first place. So yes, you can find such p and q with arbitrary large A. For example take p to be geometric upto some large $N$ and then $1\over n^3$. $q$ is geometric. Thus as $N\to\infty$ $KL(p||q_\epsilon)\to0$, but the second moment is always $+\infty$ (if you don't like this, cut $p$ at $M\gg N$. | |
| Oct 13, 2010 at 14:33 | history | edited | rasper | CC BY-SA 2.5 |
added 36 characters in body
|
| Oct 13, 2010 at 14:00 | comment | added | rasper | $p(x)$ can be part of the construction, not just $q_{\epsilon}(x)$. | |
| Oct 13, 2010 at 13:59 | history | edited | rasper | CC BY-SA 2.5 |
added 3 characters in body; edited title
|
| Oct 13, 2010 at 13:25 | history | asked | rasper | CC BY-SA 2.5 |