# Questions tagged [entropy]

The tag has no usage guidance.

314 questions
Filter by
Sorted by
Tagged with
1 vote
35 views

### Maximize differential entropy of probability distribution without fixing second moment

(Crossposted from Mathematics Stackexchange) In Christopher Bishop's book "Pattern Recognition and Machine Learning", pages 53-54 specifically, he uses Lagrange multipliers to find the ...
11 views

• 1,063
66 views

### "Calculus" of metric entropy: How does metric entropy behave with respect to binary operators?

This is a general question, more of a reference request. tl;dr: Is there a "calculus" for computing metric entropy bounds? Given a function space $\mathcal{F}$, we may define its metric ...
118 views

### Let $\mu : [0, T] \to \mathcal P_2^a (\mathbb R^d), t \mapsto \mu_t$ be absolutely continuous. Is $t \mapsto \mathcal H (\mu_t)$ continuous?

We endow the space $\mathcal P_2^a (\mathbb R^d)$ of absolutely continuous probability measures with finite second moment with the Wasserstein distance $W_2$. Let $\mathcal H (\mu)$ be the relative ...
• 1,063
1 vote
89 views

• 1,667
107 views

### Differential entropy of random Gibbs measure

There is a question I have been wondering about for a while, which I have thus far not been able to resolve. The problem revolves around random Gibbs measures. I am not very well-versed in the more ...
117 views

201 views

### How to categorify entropy/perplexity?

(Migrating from math.stackexchange.com per commenter suggestion) I was reading Baez, Fritz, and Leinster's "A Characterization of Entropy in Terms of Information Loss", and wondered if, ...
• 261
690 views

• 79
253 views

### Covering/Bracketing number of monotone functions on $\mathbb{R}$ with uniformly bounded derivatives

I am interested in the $\| \cdot \|_{\infty}$-norm bracketing number or covering number of some collection of distribution functions on $\mathbb{R}$. Let $\mathcal{F}$ consist of all distribution ...
• 93
1 vote
The following follows from Proposition 9.6 in Approximation of zonoids by zonotopes, Acta Math. 162 (1989), no. 1-2, 73–141, by Bourgain, J., Lindenstrauss, J., and Milman, V. Let $X\subset L^1(\mu)$ ...