We endow the space $\mathcal P_2^a (\mathbb R^d)$ of absolutely continuous probability measures with finite second moment with the Wasserstein distance $W_2$. Let $\mathcal H (\mu)$ be the relative entropy of $\mu$ w.r.t. Lebesgue measure. It is well-known that $\mathcal H$ is not continuous but only lower semi-continuous w.r.t. $W_2$.

Let $\mu : [0, T] \to \mathcal P_2^a (\mathbb R^d), t \mapsto \mu_t$ be absolutely continuous. Is $t \mapsto \mathcal H (\mu_t)$ continuous?

Thank you so much for your elaboration!

Update Let $(X, d)$ be a metric space. A map $y:[a, b] \rightarrow X$ is called absolutely continuous IFF there is $v \in L^1([a, b])$ such that $v \geq 0$ almost everywhere and $d(y(s), y(t)) \leq \int_s^t v(r) \, \mathrm{d} r$ for every $a \leq s \leq t \leq b$. A map $y:[0, \infty) \rightarrow X$ is called absolutely continuous on $[0, \infty)$ IFF $y|_{[a, b]}$ is absolutely continuous for any $0\le a<b$.

  • $\begingroup$ What do you mean by absolute continuity of the map $t\mapsto\mu_t$? $\endgroup$
    – R W
    Commented Oct 24, 2023 at 2:31
  • $\begingroup$ @RW Please see my update. $\endgroup$
    – Akira
    Commented Oct 24, 2023 at 6:31

1 Answer 1


$\newcommand{\R}{\mathbb R}$The answer is NO. I will provide below a counterexample in dimension $d=1$.

Preliminaries: Let's agree that the entropy is $$ H(\rho)=\int_{\mathbb R}\rho(x)\log\rho(x) dx $$ whenever $\rho=\rho(x) dx$ is absolutely continuous w.r.t. the Lebesgue measure $dx$. The counterexample to full (weak, or Wasserstein) continuity of the entropy is the typical "binary 1-0-1-0..." oscillations. I guess this is clear if you come from an information-theory background, I actually learnt this myself from here on Math Overflow. To be more precise, let $$ A_n=\bigcup_{i=0}^{n-1}\Big[2i/(2n)\,,\,(2i+1)/2n\Big) ,\qquad \rho_n=2\chi_{A_n}(x) dx $$ and $$ A=[0,1]\qquad\rho_*=\chi_{[0,1]}(x)dx. $$ You can think of this as a sequence of binary words spread across the fixed domain $x\in[0,1]$, the value $\rho_n(x)=2$ corresponding to the $1$-bit and the value $\rho_n(x)=0$ to the $0$-bit. So, we have a sequence of words $\rho_1=(1,0)$, then $\rho_2=(1,0,1,0)$, and so on, $\rho_n=\underbrace{(1,0,\dots,1,0)}_{n\text{ times}}$, each time with thinner and thinner elementary bits of width $\frac 1{2n}$. It is easy to check that $\rho_n$ converges narrowly (in duality with $C([0,1])$ test functions) to $\rho_*$, hence also in the $W_2$ distance. (Recall that, in bounded domains, convergence in any Wasserstein distance is equivalent to narrow convergence). However, one can compute explicitly $$ H(\rho_n)=n\times 2\log 2\times\frac 1{2n}=\log 2 $$ (number $n$ of bits, each of height $2$ and of width $\frac1{2n}$). This obviously does not converge to $H(\rho_*)=\int 1\log 1\,dx =0$.

The counterexample below will be constructed by suitably interpolating the $\rho_n$'s in time.

Construction of the counterexample: Pick any decreasing sequence $t_k\to 0$ with $t_1=1$, and set $h_k=t_k-t_{k+1}$. By an immediate extraction argument, there exists a subsequence $\rho_k=\rho_{n_k}$ such that $$ W_2(\rho_k,\rho_*)\leq h_k $$ and therefore by triangular inequality $$ W_2(\rho_k,\rho_{k+1})\leq h_k+h_{k+1} $$ for all $k\geq 1$. Let us now define the curve $(\rho(t))_{t\in(0,1]}$ by setting $$ t=t_k:\qquad \rho(t_k):=\rho_k $$ and $$ t\in(t_{k+1},t_k):\qquad \rho(t):=\text{the time-}h_k\text{ geodesic }W_2 \text{interpolation between }\rho_{k+1},\rho_k. $$ (I hope I don't need to make this more explicit.) By construction this curve is absolutely continuous (at least for $t>0$) with piecewise-constant metric speed $$ t\in(t_{k+1},t_k):\qquad |\dot \rho|(t) =\frac{W_2(\rho_k,\rho_{k+1})}{h_k} \leq \frac{h_k+h_{k+1}}{h_{k+1}}. $$ Moreover one also has that, for any small $t_0>0$ and with the appropiate choice of $k_0$ such that $t_0\in[t_{k_0},t_{k_0+1})$, $$ \int_{t_0}^1|\dot\rho|(t)dt \leq \sum\limits_{k=1}^{k_0}\frac{h_k+h_{k+1}}{h_k} h_k \leq \sum\limits_{k\geq 1}h_k+h_{k+1} \leq 2, $$ since by construction $\sum h_k=1$. This shows that the metric speed $t\mapsto |\dot\rho|(t)$ is globally $L^1$ up to $t_0=0^+$. By standard completeness one can extend by continuity $\rho(0)=\lim \rho(t_k)=\rho_*$ while maintaining absolute continuity (including up to $t=0$), and the thesis follows since then $$ H(\rho(t_k))=\log 2 \qquad \text{but}\qquad H(\rho(0))=0. $$

  • $\begingroup$ Thank you very much for taking time to write such a detailed answer! $\endgroup$
    – Akira
    Commented Mar 27 at 15:14
  • $\begingroup$ Errrrr.... I just realized there was a very crucial mistake in my proof, I modified to fix it using extraction of a subsequence $\endgroup$ Commented Apr 1 at 9:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.