Timeline for Continuous self-information
Current License: CC BY-SA 3.0
4 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jan 28, 2015 at 5:53 | comment | added | ASML | @kodlu: Thanks for your comment, but I'm not sure if you're right. Differential entropy is well-defined and discretization not necessary. If I sample from a multivariate Gaussian and use a non-parametric entropy estimator, the estimate converges to the expected theoretical value as the samples go to infinity. It's only the special case $H(x|x)$ that seems to be tricky for some reason. | |
| Jan 28, 2015 at 1:23 | comment | added | Vincent | It looks like the last part of the above comment is missing | |
| Jan 28, 2015 at 1:12 | comment | added | kodlu | It might help you to know that the $\Delta-$ quantization $X^{\Delta}$ of a continuous random variable $X$ obeys $H(X^{\Delta})+\log \Delta \rightarrow h(X)$ as $\Delta\rightarrow 0.$ This clarifies the relationship between discrete and continuous entropy and shows that continuous entropy without specifying accuracy is not directly relevant to practice, e.g., estimation. If you quantize to $n$ bits, the quantization has entropy $h(X)+n$ and on average requires this much information to describe. | |
| Jan 28, 2015 at 0:17 | history | asked | ASML | CC BY-SA 3.0 |