Skip to main content
4 of 4
added 13 characters in body
Arthur B
  • 1.9k
  • 1
  • 14
  • 18

A sufficient condition would be that the tails of $f$ decay as $O(x^{-1-\epsilon})$, which is most distributions you might encounter over $\mathbb{R}$.

That being said, the differential entropy of a continuous pdf isn't really a meaningful physical, or information theoretical, quantity. The equation isn't homogeneous: changing the units in which you measure $x$ changes the differential entropy (dimensional analysis can lead to a surprising amount of mathematical insights, even outside of physics).

What does make sense is the KL-divergence of one distribution with respect to another. For a pdf with a compact support, you're implicitly looking at the divergence with respect to the uniform distribution. However, there is no "uniform" distribution over the real line and thus the concept isn't meaningful.

Arthur B
  • 1.9k
  • 1
  • 14
  • 18