Skip to main content
1 of 4
Arthur B
  • 1.9k
  • 1
  • 14
  • 18

A sufficient condition would be that the tails of $f$ decay as $O(x^{-1-\epsilon})$, which is most distributions you might encounter over $\mathbb{R}$.

That being said, the entropy of a continuous pdf isn't a meaningful physical, or information theoretical, quantity. The equation isn't homogeneous, changing the units in which you measure $x$ changes the entropy (a surprising amount of mathematical insights can be had by dimensional analysis, even outside of physics).

What does make sense is the KL-divergence of one distribution with respect to another. For a pdf with a compact support, you're implicitly looking at the divergence with respect to the uniform distribution. However, there is no "uniform" distribution over the real line and thus the concept isn't meaningful.

Arthur B
  • 1.9k
  • 1
  • 14
  • 18