# Mathematical difference between entropy and energy

I have a rather soft question. Let's assume that we consider the heat equation posed in $S^1$: $$\partial_t u=\partial_x^2u.$$ It is well known that if we define the functionals $$H(t)=\int_{-\pi}^\pi u\log(u)-u+1dx,$$ $$E(t)=\int_{-\pi}^\pi u^2dx,$$ they decay. As far as I know, in the literature, a functional similar to $H$ is called entropy while a functional like $E$ is called energy. I understand the physical reasons behind that.

The question is: are there some subtle mathematical differences between these two Lyapunov functionals that we emphasize by using different names entropy/energy? Or, on the other hand, it's just a reminder from the physical origin of both terms?

• The definition of entropy is usually $-H(t)$. Apr 13, 2015 at 14:25

First, note that entropy is well-defined only if $u$ is positive. Using integration by parts, it is in fact true on the flat torus or $\mathbb{R}^n$ (if $u$ is nonnegative and decays in space fast enough) that $$\frac{\partial}{\partial t}\frac{1}{p(p-1)}\log \int u^p = -\frac{\int u^{p-2}|\nabla u|^2}{\int u^p}\le 0$$ for all $p > 1$. The case $p = 2$ is the usual $L^2$ energy inequality. If you normalize $u$ so that $$\int u = 1$$ and take the limit $p \rightarrow 1$, you get the entropy inequality. So there are $L^p$ analogues of energy or entropy, depending on your point of view.

In information theory, if $u$ is a probability density, then $$-\int u\log u$$ is called Shannon entropy and $$\frac{1}{1-p}\log\int u^p$$ is called Rényi entropy.

• Thank you for your answer! I was assuming $u_0$, to be positive, so there was no problem on difining the entropy. Why do you say the case $2=p$ is the usual energy inequality? Maybe is a stupid question, but, even if I see that this quotient has the same flavour, I don't see why it's the same thing. Apr 13, 2015 at 15:58
• $\int u^2$ decays in time if and only if $\frac{1}{2}\log \int u^2$ does. Apr 13, 2015 at 17:37
• An aside: The $L^p$ inequality is used for Moser iteration, which establishes an a priori estimate for $u$. The basic idea is to rewrite the negative term (up to a constant factor depending on $p$) as $-\int |\nabla u^{p/2}|^2$ and apply the Sobolev inequality. This gives you an estimate for a higher $L^p$ norm of $u$ in terms of a lower one. Repeat and show that the accumulation of constant factors is bounded as $p \rightarrow \infty$. In the limit, you get an $L^\infty$ bound on $u$ in terms of an $L^p$ bound (where $p > 1$). This works for a variable coefficient heat equation. Apr 14, 2015 at 12:53
• where can I see this computation? Can you give me a reference? Apr 14, 2015 at 15:13
• I think if you google "Moser iteration parabolic PDE", you'll find lots of presentations on it. They will also cite Moser's original papers. Apr 14, 2015 at 17:32

I would personnally say that physics is all what lies behind the names. One slight difference between them is that, at least in the field of PDEs, whatever we call an entropy is always monotone along the flow of the equation (and usually made positive decreasing up to some constants in its definition) whereas an energy has a broader meaning ; one sometimes hears terms like "$H^s$ energy", "higher [order] energy", "$L^p$ energy [estimate]" and so on. Those quantities do not necessarily decrease ; instead, they may very well increase, as do the $H^s$ energies for various instances of the NLS (non linear Schrödinger) equation. They could exhibit some weirder behaviour, like the usual $L^2$ energy for the Euler equation (I'm talking here about the work of De Lellis and Székelyhidi).

• Thank you for your answer! Actually, there are examples where $H$ does not decay (for instance, the parabolic-elliptic) Keller-Segel. Also, I would say that the regularity to preserve $L^2$ for the Euler flow and the regularity to preserve $LlogL$ is the same, $C^{1/3}$. Isn't it? With this I meant that there should be something else, not merely the monotonicity or well-behaviour. Am I right? Apr 13, 2015 at 16:02

It is probably worth mentioning that the the heat equation is inherently linked to energy and entropy in two ways:

1. The heat equation is the gradient flow of the energy in the Hilbert space $L^2$ (classical).

2. The heat equation is the "gradient flow of the entropy in Wasserstein space $W_2$" (Jordan-Kinderlehrer-Otto, also Ambrosio-Gigli-Savare).

Unfortunately, the PDE community has thoroughly confused this issue by using the word entropy for what physicists would call free energy. Therefore, if you are a physicist, entropy increases, but if you work on hyperbolic conservation laws, entropy decreases!

• Well, given that entropy is supposed to measure how much randomness or noise there is, it is natural to have it increase in time. So the hyperbolic conservation law people somehow got it wrong. Apr 13, 2015 at 20:23
• Thank you for your comment. I already knew about the term "free energy". There is another question arising here: what is the difference between and energy and a free energy? Apr 13, 2015 at 21:18
• You will find free energy explained in any basic text on thermodynamics Apr 13, 2015 at 23:04