Assume we have two sampling process, i.e. we can draw samples from two (not explicitly known) distributions P and Q. Is there any simple way to calculate the KL-divergence D(P||Q)? P and Q could be discrete or continuous.
$\begingroup$
$\endgroup$
1
-
$\begingroup$ This answer stats.stackexchange.com/questions/211175/… and references therein is relevant. $\endgroup$– kjetil b halvorsenCommented Jul 27, 2017 at 14:42
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
1
The whole paper here is on that topic
-
1$\begingroup$ The linked paper does not address the original question of two-sample estimation of KL-divergence, but its references [3]-[5] may. Currently the link is dead. I have submitted an edit to update it to web.archive.org/web/20130305001428/http://cosmal.ucsd.edu/~gert/… $\endgroup$ Commented Feb 10, 2016 at 20:40
$\begingroup$
$\endgroup$
0
Maybe this link is more stable? http://eceweb.ucsd.edu/~gert/papers/isit_2010.pdf