Timeline for A necessary condition for differential entropy to be finite
Current License: CC BY-SA 3.0
11 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Mar 21, 2017 at 23:51 | comment | added | Deane Yang | I meant differential entropy. You can check that if $f$ is the probability density for a random vector $X \in \mathbb{R}^n$, then $N[X] = \exp -\int f\log f$ scales like $N[tX] = t^nN[X]$. It has units of volume. | |
| Mar 21, 2017 at 23:48 | history | edited | Arthur B | CC BY-SA 3.0 |
added 13 characters in body
|
| Mar 21, 2017 at 23:47 | comment | added | Arthur B | Entropy is, differential entropy isn't. | |
| Mar 21, 2017 at 23:43 | comment | added | Deane Yang | Entropy is in fact a meaningful physical or information quantity in the same sense that decibels are. Both are logarithmic quantities. The entropy power ($exp -\int f\log f$) scales quite nicely. | |
| Mar 21, 2017 at 22:17 | history | edited | Arthur B | CC BY-SA 3.0 |
added 20 characters in body
|
| Mar 21, 2017 at 19:06 | comment | added | Henry.L | If $f$ is the density of the measure w.r.t. the Lebesgue measure, then this seems pretty restrictive. Is there a necessary condition? | |
| Mar 21, 2017 at 18:21 | comment | added | Arthur B | Also, by that definition, you can get negative entropy, even though the entropy is supposed to be the log of a number of configurations. Yes you can define it mathematically, but how meaningful is it? | |
| Mar 21, 2017 at 18:12 | comment | added | Arthur B | It's a matter of philosophy. Accepting these priors give you probability paradoxes. | |
| Mar 21, 2017 at 17:49 | comment | added | Qiaochu Yuan | The concept is perfectly meaningful. You can take the KL divergence of a measure, not necessarily a probability measure, with respect to another measure as long as the relevant Radon-Nikodym derivative is defined. The uniform distribution over the real line corresponds to Lebesgue measure, and can be used e.g. as an "improper prior" in Bayesian inference. | |
| Mar 21, 2017 at 17:12 | history | edited | Arthur B | CC BY-SA 3.0 |
deleted 2 characters in body
|
| Mar 21, 2017 at 17:06 | history | answered | Arthur B | CC BY-SA 3.0 |