4
$\begingroup$

For Lebesgue-absolutely continuous probability measures $\rho\ll \mathcal{L}^d$ in the whole space $\mathbb{R}^d$ with finite second moments (i-e $\rho\in \mathcal{P}^2_{ac}(\mathbb{R}^d)$), let $$ \mathcal{H}(\rho)=\int_{\mathbb{R}^d}\rho\log \rho \,dx $$ be the Boltzmann entropy. It is well-known that $\mathcal{H}$ is lower semi-continuous (l.s.c.) in the Wasserstein metric $W_2$.

Question: Fix $C>0$. Is the sublevel set $\{\rho \in \mathcal{P}^2_{ac}(\mathbb{R}^d) : \mathcal{H}(\rho) \le C\}$ compact in $W_2$?

Thank you so much for your elaboration! Any reference is greatly approciated!

$\endgroup$
8
  • 1
    $\begingroup$ Clearly not, since you can take $\rho_n(x)=\rho(x-x_n)$ with $x_n\to\infty$, and this has a fixed entropy but no convergent subsequence. Also, is you formula for entropy missing a - sign? And shouldn't you call $\rho$ densities rather than measures? $\endgroup$
    – Kostya_I
    Commented Nov 2, 2023 at 12:47
  • 1
    $\begingroup$ @Kostya_I Thank you so much for your help! I follow the convention in optimal transport that the Boltzmann entropy is non-negative. In $\mathcal{P}^2_{ac}(\mathbb{R}^d)$, I identify a measure with its density. $\endgroup$
    – Akira
    Commented Nov 2, 2023 at 12:51
  • $\begingroup$ Hi Akira, do you see many people call this Boltzmann entropy ? I'd say "differential entropy", "Gibbs entropy", or just "entropy",. I think that Boltzmann was only interested in uniform measures -or rather trivial measures, on a single microstate, as the formula engraved on his tombstone indicates. I don't think this set is compact: take 2 0-entropy (Dirac) measures, $C=0$, make them move apart on $\mathbb R$, their distance tends to infinity. You can do the same with any single probability measure. You may also make up examples with classical distributions whose entropies are known and match. $\endgroup$
    – plm
    Commented Nov 2, 2023 at 12:56
  • $\begingroup$ @Akira, the entropy in your formula is not necessarily non-negative, e.g., if the density is smaller than 1 everywhere, then the integrand is negative. $\endgroup$
    – Kostya_I
    Commented Nov 2, 2023 at 13:58
  • 2
    $\begingroup$ @plm, sorry, I don't follow. If $\rho(x)$ is a probability density, then so is $\rho_a(x)=a^d\rho(ax)$ for any $a>0$. We then have $\mathcal{H}(\rho_a)=\mathcal{H}(\rho)-d\log a$. For large $a$, it tends to $+\infty$, while for small $a$, it tends to $-\infty$. $\endgroup$
    – Kostya_I
    Commented Nov 2, 2023 at 15:17

2 Answers 2

2
$\begingroup$

$\newcommand{\R}{\mathbb R}\newcommand{\H}{\mathcal H}$Building up on previous comments, and slighlty elaborating. The answer to your question, as is, is NO. Recall that convergence in the Wasserstein distance $W_2$ is equivalent to weak-* convergence (duality with continuous bounded functions) and convergence of the second moments (or equivalently, with all continuous functions growing at most quadratically at infinity). As a consequence, in order for your compactness to hold you would need to control the second moments in sublevelsets. However, the entropy is translation-invariant, so mass can escape at infinity in a sublevelset $\{\H\leq C\}$. More precisely, for a counterexample take as in @Kostya_I 's comment any measure $\rho$ with finite entropy $\H(\rho)=C<\infty$, take any $x_n\in\R^d$ such that $|x_n|\to\infty$, and consider the translated sequence $\rho_n=\rho(\cdot-x_n)$. Then obviously $\H(\rho_n)=\H(\rho)=C$ for any $n$, but $\rho_n$ fails to converge in the Wasserstein distance (since for example $\rho_n$ converges weakly to zero when tested with respect to any compactly supported function).

Since the problem here is only due mass escaping at infinity, for your desired compactness to hold you only need a little bit of confinement. For example it would suffice to control the second moment $\mathfrak m_2(\rho)=\int|x|^2\rho(dx)$, for example by adding a potential energy $$ \mathcal F(\rho)=\H(\rho)+\int V(x)\rho(dx) $$ for some potential $V(x)\gtrsim C|x|^2$ growing at least quadratically at infinity. For this you can use the handy Carleman estimate $$ \H(\rho)\geq -C_\alpha(1+\mathfrak{m}_2(\rho))^\alpha, \qquad \alpha\in \big(d/(d+2),1\big). $$ Check the original JKO paper [1], proof of proposition 4.1 (eqs. 14 and 15 therein). Actually you can replace the potential energy $\int V\rho$ by anything controlling $\mathfrak m_2(\rho)^{\frac{d}{d+2}}$ in order to retrive control over the moments from energy bounds $\mathcal F(\rho)\leq C\Rightarrow \mathfrak m_2(\rho)\leq M(C)$.

Connecting with @pseudocydonia 's answer above: another way to interpret the "potential energy above" would be to consider instead the relative entropy $\H_\mu(\rho)=\int \frac{\rho}{\mu}\log\left(\frac{\rho}{\mu}\right)d\mu$ for a log-quadratic reference measure $\mu\sim e^{-V}$ with $V\gtrsim C|x|^2$.


Side note: in your setting you also have a slightly worse problem than what you may suspect. When computed relatively to the Lebesgue measure on (sufficiently wide) unbounded sets the entropy is actually not bounded from below on $\mathcal P_{ac}^2$.


[1] Jordan, R., Kinderlehrer, D., & Otto, F. (1998). The variational formulation of the Fokker--Planck equation. SIAM journal on mathematical analysis, 29(1), 1-17.

$\endgroup$
1
$\begingroup$

It is known that the sublevel sets of the relative entropy are tight when the reference measure is finite, and in fact are also compact in the topology of setwise convergence (which is stronger than the weak* topology but weaker than the strong topology), see Remark 7.3 in "Calculus and heat flow in metric measure spaces and applications to spaces with Ricci bounds from below" by AGS: https://arxiv.org/pdf/1106.2090.pdf. (Their argument is carried out on a more general metric measure space than what you ask for.) For the case of a sigma-finite reference measure, as you consider, I expect the answer is also positive but one would have to carry out a change-of-reference-measure argument carefully.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.