Timeline for Bounding Entropy in terms of KL-Divergence
Current License: CC BY-SA 3.0
4 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jun 14, 2013 at 18:34 | answer | added | David Reeb | timeline score: 4 | |
| May 21, 2012 at 15:53 | comment | added | Ben Charrow | Thanks for the bounds! What notation would you use for cross-entropy? As far as I can tell, there isn't an established standard. I know that I'm overloading $H(X,Y)$, but this is what I've seen elsewhere. | |
| May 19, 2012 at 6:35 | comment | added | Ashok | There are some issues with your notation and definition. $H(X,Y)$ usually denoted the joint entropy of $X$ and $Y$ and $KL(X\|Y)=\int f\log (f/g)=\int f\log f-\int f\log g$ when the integrals exist. With the given information one can say that $$|H(X)-H(Y)|<\alpha+\int f\log g+H(Y) \text{ when } H(X)\le H(Y)$$ and $$|H(X)-H(Y)|>-\alpha-\int f\log g-H(Y) \text{ when } H(X)>H(Y)$$ | |
| May 18, 2012 at 16:48 | history | asked | Ben Charrow | CC BY-SA 3.0 |