Return to Question

An ensemble corresponding to a probability distribution usually has finite free energy so it is not a great loss of generality to assume that the ensemble also has finite energy in following discussions.

It is known that when the probability distribution $\mu$(assuming it is also dominated by Lebesgue measure and its density $f_{\mu}(x)$ for simplicity.) has a finite (compact) support then the classical differential entropy $$En(\mu) = \int_{\mathbb{R}} -\log[f_{\mu}(x)]f_{\mu}dx$$ can be bounded by Bekenstein bound (Wiki) and hence finite.

However, when the support of $\mu$ is not finite (compact), there exists counter example that the entropy could be infinite (Math.SE).

Question:

(1)Is finite support of $\mu$ also a necessary condition to make sure the $\mu$ has a finite entropy?

Update: It is clear that a $\mu$ with infinite support can have finite entropy. Thanks Anthony Quas for pointing out.

(2)Is there a characterization(sufficient and necessary condition) of probability distributions/statistical ensembles with finite entropy?

(3)Also, it is known that Bekenstein bound may also be applied(with volumes defined only for atoms) to entropy defined for $\sigma$-algebras(MO.post), so can we translate the characterization in (2) onto $\sigma$-algebras?

(4)From physicists' viewpoint, what will a finite entropy system looks like?

An ensemble corresponding to a probability distribution usually has finite free energy so it is not a great loss of generality to assume that the ensemble also has finite energy in following discussions.

It is known that when the probability distribution $\mu$(assuming it is also dominated by Lebesgue measure and its density $f_{\mu}(x)$ for simplicity.) has a finite (compact) support then the classical differential entropy $$En(\mu) = \int_{\mathbb{R}} -\log[f_{\mu}(x)]f_{\mu}dx$$ can be bounded by Bekenstein bound (Wiki) and hence finite.

However, when the support of $\mu$ is not finite (compact), there exists counter example that the entropy could be infinite (Math.SE).

Question:

(1)Is finite support of $\mu$ also a necessary condition to make sure the $\mu$ has a finite entropy?

Update: It is clear that a $\mu$ with infinite support can have finite entropy. Thanks Anthony Quas for pointing out.

(2)Is there a characterization(sufficient and necessary condition) of probability distributions/statistical ensembles with finite entropy?

(3)Also, it is known that Bekenstein bound may also be applied(with volumes defined only for atoms) to entropy defined for $\sigma$-algebras(MO.post), so can we translate the characterization in (2) onto $\sigma$-algebras?

(4)From physicists' viewpoint, what will a finite entropy system looks like?

An ensemble corresponding to a probability distribution usually has finite free energy so it is not a great loss of generality to assume that the ensemble also has finite energy in following discussions.

It is known that when the probability distribution $\mu$(assuming it is also dominated by Lebesgue measure and its density $f_{\mu}(x)$ for simplicity.) has a finite (compact) support then the classical differential entropy $$En(\mu) = \int_{\mathbb{R}} -\log[f_{\mu}(x)]f_{\mu}dx$$ can be bounded by Bekenstein bound (Wiki) and hence finite.

However, when the support of $\mu$ is not finite (compact), there exists counter example that the entropy could be infinite (Math.SE).

Question:

(1)Is finite support of $\mu$ also a necessary condition to make sure the $\mu$ has a finite entropy?

Update: It is clear that a $\mu$ with infinite support can have finite entropy. Thanks Anthony Quas for pointing out.

(2)Is there a characterization(sufficient and necessary condition) of probability distributions/statistical ensembles with finite entropy?

(3)Also, it is known that Bekenstein bound may also be applied(with volumes defined only for atoms) to entropy defined for $\sigma$-algebras(MO.post), so can we translate the characterization in (2) onto $\sigma$-algebras?

(4)From physicists' viewpoint, what will a finite entropy system looks like?

An ensemble corresponding to a probability distribution usually has finite free energy so it is not a great loss of generality to assume that the ensemble also has finite energy in following discussions.

It is known that when the probability distribution $\mu$(assuming it is also dominated by Lebesgue measure and its density $f_{\mu}(x)$ for simplicity.) has a finite (compact) support then the classical differential entropy $$En(\mu) = \int_{\mathbb{R}} -\log[f_{\mu}(x)]f_{\mu}dx$$ can be bounded by Bekenstein bound (Wiki) and hence finite.

However, when the support of $\mu$ is not finite (compact), there exists counter example that the entropy could be infinite (Math.SE).

Question:

(1)Is finite support of $\mu$ also a necessary condition to make sure the $\mu$ has a finite entropy?

Update: It is clear that a $\mu$ with infinite support can have finite entropy. Thanks Anthony Quas for pointing out.

(2)Is there a characterization(sufficient and necessary condition) of probability distributions/statistical ensembles with finite entropy?

(3)Also, it is known that Bekenstein bound may also be applied(with volumes defined only for atoms) to entropy defined for $\sigma$-algebras(MO.post), so can we translate the characterization in (2) onto $\sigma$-algebras?

(4)From physicists' viewpoint, what will a finite entropy system looks like?

An ensemble corresponding to a probability distribution usually has finite free energy so it is not a great loss of generality to assume that the ensemble also has finite energy in following discussions.

It is known that when the probability distribution $\mu$(assuming it is also dominated by Lebesgue measure and its density $f_{\mu}(x)$ for simplicity.) has a finite (compact) support then the classical differential entropy $$En(\mu) = \int_{\mathbb{R}} -\log[f_{\mu}(x)]f_{\mu}dx$$ can be bounded by Bekenstein bound (Wiki) and hence finite.

However, when the support of $\mu$ is not finite (compact), there exists counter example that the entropy could be infinite (Math.SE).

Question:

(1)Is finite support of $\mu$ also a necessary condition to make sure the $\mu$ has a finite entropy?

(2)Is there a characterization(sufficient and necessary condition) of probability distributions/statistical ensembles with finite entropy?

(3)Also, it is known that Bekenstein bound may also be applied(with volumes defined only for atoms) to entropy defined for $\sigma$-algebras(MO.post), so can we translate the characterization in (2) onto $\sigma$-algebras?

(4)From physicists' viewpoint, what will a finite entropy system looks like?

An ensemble corresponding to a probability distribution usually has finite free energy so it is not a great loss of generality to assume that the ensemble also has finite energy in following discussions.

It is known that when the probability distribution $\mu$(assuming it is also dominated by Lebesgue measure and its density $f_{\mu}(x)$ for simplicity.) has a finite (compact) support then the classical differential entropy $$En(\mu) = \int_{\mathbb{R}} -\log[f_{\mu}(x)]f_{\mu}dx$$ can be bounded by Bekenstein bound (Wiki) and hence finite.

However, when the support of $\mu$ is not finite (compact), there exists counter example that the entropy could be infinite (Math.SE).

Question:

(1)Is finite support of $\mu$ also a necessary condition to make sure the $\mu$ has a finite entropy?

Update: It is clear that a $\mu$ with infinite support can have finite entropy. Thanks Anthony Quas for pointing out.

(2)Is there a characterization(sufficient and necessary condition) of probability distributions/statistical ensembles with finite entropy?

(3)Also, it is known that Bekenstein bound may also be applied(with volumes defined only for atoms) to entropy defined for $\sigma$-algebras(MO.post), so can we translate the characterization in (2) onto $\sigma$-algebras?

(4)From physicists' viewpoint, what will a finite entropy system looks like?