Timeline for Property of relative entropy [closed]
Current License: CC BY-SA 3.0
11 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Dec 7, 2014 at 19:03 | history | closed |
Denis Serre R W Yemon Choi Peter Michor Ryan Budney |
Not suitable for this site | |
| Dec 5, 2014 at 14:32 | comment | added | Sedi | Dear Pablo, Thank you for your wishes. All your references are in finite space. I haven't seen in general form. Anyways, its maybe correct, I need to study more. | |
| Dec 5, 2014 at 13:29 | comment | added | Pablo Lessa | Sedi: There's no real issue with the probability space being infinite you just need to study up a bit. The proof Michele gives below works verbatim. Jensen's inequality is valid for general probability measures. Check out the wikipedia page for Jensen's inequality or any book on information theory (e.g. Gray's book Chapter 7, lemma 7.1). I don't think this question is appropriate for mathoverflow since it's treated in detail in practically any reference on the subject. Best of luck. | |
| Dec 5, 2014 at 13:15 | comment | added | Sedi | Dear Pablo, According to the link which you have sent, there Gibb's inequality is used and in Gibb's inequality they have used that finite point in sample space. That is my problem, if it is infinite then what is the condition. | |
| Dec 5, 2014 at 12:22 | comment | added | Pablo Lessa | As Michele shows in his answer the divergence is zero if and only if the two probabilities are equal. This is an elementary property of KL-Divergence (in fact Michele gave the full proof). Have a look at the wikipedia page and any book on the subject (e.g. Gray's book on information theory): en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence | |
| S Dec 5, 2014 at 12:19 | history | edited | Sedi | CC BY-SA 3.0 |
corrected spelling
|
| Dec 5, 2014 at 12:14 | review | Suggested edits | |||
| S Dec 5, 2014 at 12:19 | |||||
| Dec 5, 2014 at 11:56 | answer | added | user47274 | timeline score: 2 | |
| Dec 5, 2014 at 11:30 | review | Close votes | |||
| Dec 7, 2014 at 19:03 | |||||
| Dec 5, 2014 at 11:20 | history | edited | Sedi | CC BY-SA 3.0 |
added 404 characters in body
|
| Dec 5, 2014 at 10:40 | history | asked | Sedi | CC BY-SA 3.0 |