Skip to main content
edited body
Source Link
Deane Yang
  • 27.5k
  • 5
  • 89
  • 180

First, note that entropy is well-defined only if $u$ is positive. Using integration by parts, it is in fact true on the flat torus or $\mathbb{R}^n$ (if $u$ is nonnegative and decays in space fast enough) that $$ \frac{\partial}{\partial t}\frac{1}{p}\log \int u^p = -(p-1)\frac{\int u^{p-2}|\nabla u|^2}{\int u^p}\le 0 $$$$ \frac{\partial}{\partial t}\frac{1}{p(p-1)}\log \int u^p = -\frac{\int u^{p-2}|\nabla u|^2}{\int u^p}\le 0 $$ for all $p > 1$. The case $p = 2$ is the usual $L^2$ energy inequality. If you normalize $u$ so that $$ \int u = 1 $$ and take the limit $p \rightarrow 1$, you get the entropy inequality. So there are $L^p$ analogues of energy or entropy, depending on your point of view.

In information theory, if $u$ is a probability density, then $$ -\int u\log u $$ is called Shannon entropy and $$ \frac{1}{1-p}\int u^p $$$$ \frac{1}{1-p}\log\int u^p $$ is called Rényi entropy.

First, note that entropy is well-defined only if $u$ is positive. Using integration by parts, it is in fact true on the flat torus or $\mathbb{R}^n$ (if $u$ is nonnegative and decays in space fast enough) that $$ \frac{\partial}{\partial t}\frac{1}{p}\log \int u^p = -(p-1)\frac{\int u^{p-2}|\nabla u|^2}{\int u^p}\le 0 $$ for all $p > 1$. The case $p = 2$ is the usual $L^2$ energy inequality. If you normalize $u$ so that $$ \int u = 1 $$ and take the limit $p \rightarrow 1$, you get the entropy inequality. So there are $L^p$ analogues of energy or entropy, depending on your point of view.

In information theory, if $u$ is a probability density, then $$ -\int u\log u $$ is called Shannon entropy and $$ \frac{1}{1-p}\int u^p $$ is called Rényi entropy.

First, note that entropy is well-defined only if $u$ is positive. Using integration by parts, it is in fact true on the flat torus or $\mathbb{R}^n$ (if $u$ is nonnegative and decays in space fast enough) that $$ \frac{\partial}{\partial t}\frac{1}{p(p-1)}\log \int u^p = -\frac{\int u^{p-2}|\nabla u|^2}{\int u^p}\le 0 $$ for all $p > 1$. The case $p = 2$ is the usual $L^2$ energy inequality. If you normalize $u$ so that $$ \int u = 1 $$ and take the limit $p \rightarrow 1$, you get the entropy inequality. So there are $L^p$ analogues of energy or entropy, depending on your point of view.

In information theory, if $u$ is a probability density, then $$ -\int u\log u $$ is called Shannon entropy and $$ \frac{1}{1-p}\log\int u^p $$ is called Rényi entropy.

added 70 characters in body
Source Link
Deane Yang
  • 27.5k
  • 5
  • 89
  • 180

First, note that entropy is well-defined only if $u$ is positive. Using integration by parts, it is in fact true on the flat torus or $\mathbb{R}^n$, if (if $u$ is nonnegative and decays in space fast enough,) that $$ \frac{\partial}{\partial t}\frac{1}{p}\log \int u^p \le 0 $$$$ \frac{\partial}{\partial t}\frac{1}{p}\log \int u^p = -(p-1)\frac{\int u^{p-2}|\nabla u|^2}{\int u^p}\le 0 $$ for all $p > 1$. The case $p = 2$ is the usual $L^2$ energy inequality. If you normalize $u$ so that $$ \int u = 1 $$ and take the limit $p \rightarrow 1$, you get the entropy inequality. So there are $L^p$ analogues of energy or entropy, depending on your point of view.

In information theory, if $u$ is a probability density, then $$ -\int u\log u $$ is called Shannon entropy and $$ \frac{1}{1-p}\int u^p $$ is called Rényi entropy.

First, note that entropy is well-defined only if $u$ is positive. Using integration by parts, it is in fact true on the flat torus or $\mathbb{R}^n$, if $u$ decays in space fast enough, that $$ \frac{\partial}{\partial t}\frac{1}{p}\log \int u^p \le 0 $$ for all $p > 1$. If you normalize $u$ so that $$ \int u = 1 $$ and take the limit $p \rightarrow 1$, you get the entropy inequality. So there are $L^p$ analogues of energy or entropy, depending on your point of view.

First, note that entropy is well-defined only if $u$ is positive. Using integration by parts, it is in fact true on the flat torus or $\mathbb{R}^n$ (if $u$ is nonnegative and decays in space fast enough) that $$ \frac{\partial}{\partial t}\frac{1}{p}\log \int u^p = -(p-1)\frac{\int u^{p-2}|\nabla u|^2}{\int u^p}\le 0 $$ for all $p > 1$. The case $p = 2$ is the usual $L^2$ energy inequality. If you normalize $u$ so that $$ \int u = 1 $$ and take the limit $p \rightarrow 1$, you get the entropy inequality. So there are $L^p$ analogues of energy or entropy, depending on your point of view.

In information theory, if $u$ is a probability density, then $$ -\int u\log u $$ is called Shannon entropy and $$ \frac{1}{1-p}\int u^p $$ is called Rényi entropy.

Source Link
Deane Yang
  • 27.5k
  • 5
  • 89
  • 180

First, note that entropy is well-defined only if $u$ is positive. Using integration by parts, it is in fact true on the flat torus or $\mathbb{R}^n$, if $u$ decays in space fast enough, that $$ \frac{\partial}{\partial t}\frac{1}{p}\log \int u^p \le 0 $$ for all $p > 1$. If you normalize $u$ so that $$ \int u = 1 $$ and take the limit $p \rightarrow 1$, you get the entropy inequality. So there are $L^p$ analogues of energy or entropy, depending on your point of view.