Skip to main content
replaced http://mathoverflow.net/ with https://mathoverflow.net/
Source Link

Try: $-H(Z) = \mathbb{E} [\log Z] $ we have an equality:

$$ H(Z) = -\mathbb{E} [\log \tfrac{1}{K}] - \alpha \mathbb{E} [\log X]-(1-\alpha) \mathbb{E} [\log Y] = \log K + \alpha H(X) +(1-\alpha) H(Y)$$

rearrange a bit to look like the original problem:

$$ \alpha H(X) +(1-\alpha) H(Y) = H(Z) - \log K $$

$K \leq 1$ using Hölder inequality and so $H(Z) \geq 0$ and $\log K < 0$ is negative, as in a relatedrelated question on entropy inequalities

$$ H(Z) - \log K \geq H(Z) \geq K^2H(Z)$$


This is not correct as stated. In fact, $-H(Z) = \mathbb{E}_Z [\log Z]$ with respect to the probability measure $Z$.

Entropy $H$ is a concave functional of the measures:

$$ H( t X + (1-t)Y) \geq H(t X + (1-t)Y|T) = tH(X) + (1-t)H(Y) $$

here $T$ is a 0-1 Bernoulli random variable with $\mathbb{P}(T=1) = t$.


I am really going to drop it after this. The way to prove one entropy is lower than another is to use conditional entropy: $H(X) \geq H(X|Y)$

What to condition on? I use unorthodox and possibly wrong notation:

$$ tH(X) + (1-t)H(Y) = \mathbb{E}[t \log X + (1-t)\log Y | X,Y,t]$$

Given distributions $X,Y$ and Bernoulli random variable $t$ we can construct the above entropy.

$$ K^2H(Z) = \mathbb{E}[\sqrt{XY}\,\big|i]^2 \mathbb{E}[t \log X + (1-t)\log Y \,\big| Z=\tfrac{\sqrt{XY}}{K},t]$$

Where $i$ is uniform in $[1,2,\dots, n]$. It still doesn't look right.

Try: $-H(Z) = \mathbb{E} [\log Z] $ we have an equality:

$$ H(Z) = -\mathbb{E} [\log \tfrac{1}{K}] - \alpha \mathbb{E} [\log X]-(1-\alpha) \mathbb{E} [\log Y] = \log K + \alpha H(X) +(1-\alpha) H(Y)$$

rearrange a bit to look like the original problem:

$$ \alpha H(X) +(1-\alpha) H(Y) = H(Z) - \log K $$

$K \leq 1$ using Hölder inequality and so $H(Z) \geq 0$ and $\log K < 0$ is negative, as in a related question on entropy inequalities

$$ H(Z) - \log K \geq H(Z) \geq K^2H(Z)$$


This is not correct as stated. In fact, $-H(Z) = \mathbb{E}_Z [\log Z]$ with respect to the probability measure $Z$.

Entropy $H$ is a concave functional of the measures:

$$ H( t X + (1-t)Y) \geq H(t X + (1-t)Y|T) = tH(X) + (1-t)H(Y) $$

here $T$ is a 0-1 Bernoulli random variable with $\mathbb{P}(T=1) = t$.


I am really going to drop it after this. The way to prove one entropy is lower than another is to use conditional entropy: $H(X) \geq H(X|Y)$

What to condition on? I use unorthodox and possibly wrong notation:

$$ tH(X) + (1-t)H(Y) = \mathbb{E}[t \log X + (1-t)\log Y | X,Y,t]$$

Given distributions $X,Y$ and Bernoulli random variable $t$ we can construct the above entropy.

$$ K^2H(Z) = \mathbb{E}[\sqrt{XY}\,\big|i]^2 \mathbb{E}[t \log X + (1-t)\log Y \,\big| Z=\tfrac{\sqrt{XY}}{K},t]$$

Where $i$ is uniform in $[1,2,\dots, n]$. It still doesn't look right.

Try: $-H(Z) = \mathbb{E} [\log Z] $ we have an equality:

$$ H(Z) = -\mathbb{E} [\log \tfrac{1}{K}] - \alpha \mathbb{E} [\log X]-(1-\alpha) \mathbb{E} [\log Y] = \log K + \alpha H(X) +(1-\alpha) H(Y)$$

rearrange a bit to look like the original problem:

$$ \alpha H(X) +(1-\alpha) H(Y) = H(Z) - \log K $$

$K \leq 1$ using Hölder inequality and so $H(Z) \geq 0$ and $\log K < 0$ is negative, as in a related question on entropy inequalities

$$ H(Z) - \log K \geq H(Z) \geq K^2H(Z)$$


This is not correct as stated. In fact, $-H(Z) = \mathbb{E}_Z [\log Z]$ with respect to the probability measure $Z$.

Entropy $H$ is a concave functional of the measures:

$$ H( t X + (1-t)Y) \geq H(t X + (1-t)Y|T) = tH(X) + (1-t)H(Y) $$

here $T$ is a 0-1 Bernoulli random variable with $\mathbb{P}(T=1) = t$.


I am really going to drop it after this. The way to prove one entropy is lower than another is to use conditional entropy: $H(X) \geq H(X|Y)$

What to condition on? I use unorthodox and possibly wrong notation:

$$ tH(X) + (1-t)H(Y) = \mathbb{E}[t \log X + (1-t)\log Y | X,Y,t]$$

Given distributions $X,Y$ and Bernoulli random variable $t$ we can construct the above entropy.

$$ K^2H(Z) = \mathbb{E}[\sqrt{XY}\,\big|i]^2 \mathbb{E}[t \log X + (1-t)\log Y \,\big| Z=\tfrac{\sqrt{XY}}{K},t]$$

Where $i$ is uniform in $[1,2,\dots, n]$. It still doesn't look right.

added 595 characters in body
Source Link
john mangual
  • 22.8k
  • 4
  • 63
  • 172

Try: $-H(Z) = \mathbb{E} [\log Z] $ we have an equality:

$$ H(Z) = -\mathbb{E} [\log \tfrac{1}{K}] - \alpha \mathbb{E} [\log X]-(1-\alpha) \mathbb{E} [\log Y] = \log K + \alpha H(X) +(1-\alpha) H(Y)$$

rearrange a bit to look like the original problem:

$$ \alpha H(X) +(1-\alpha) H(Y) = H(Z) - \log K $$

$K \leq 1$ using Hölder inequality and so $H(Z) \geq 0$ and $\log K < 0$ is negative, as in a related question on entropy inequalities

$$ H(Z) - \log K \geq H(Z) \geq K^2H(Z)$$


This is not correct as stated. In fact, $-H(Z) = \mathbb{E}_Z [\log Z]$ with respect to the probability measure $Z$.

Entropy $H$ is a concave functional of the measures:

$$ H( t X + (1-t)Y) \geq H(t X + (1-t)Y|T) = tH(X) + (1-t)H(Y) $$

here $T$ is a 0-1 Bernoulli random variable with $\mathbb{P}(T=1) = t$.


I am really going to drop it after this. The way to prove one entropy is lower than another is to use conditional entropy: $H(X) \geq H(X|Y)$

What to condition on? I use unorthodox and possibly wrong notation:

$$ tH(X) + (1-t)H(Y) = \mathbb{E}[t \log X + (1-t)\log Y | X,Y,t]$$

Given distributions $X,Y$ and Bernoulli random variable $t$ we can construct the above entropy.

$$ K^2H(Z) = \mathbb{E}[\sqrt{XY}\,\big|i]^2 \mathbb{E}[t \log X + (1-t)\log Y \,\big| Z=\tfrac{\sqrt{XY}}{K},t]$$

Where $i$ is uniform in $[1,2,\dots, n]$. It still doesn't look right.

Try: $-H(Z) = \mathbb{E} [\log Z] $ we have an equality:

$$ H(Z) = -\mathbb{E} [\log \tfrac{1}{K}] - \alpha \mathbb{E} [\log X]-(1-\alpha) \mathbb{E} [\log Y] = \log K + \alpha H(X) +(1-\alpha) H(Y)$$

rearrange a bit to look like the original problem:

$$ \alpha H(X) +(1-\alpha) H(Y) = H(Z) - \log K $$

$K \leq 1$ using Hölder inequality and so $H(Z) \geq 0$ and $\log K < 0$ is negative, as in a related question on entropy inequalities

$$ H(Z) - \log K \geq H(Z) \geq K^2H(Z)$$


This is not correct as stated. In fact, $-H(Z) = \mathbb{E}_Z [\log Z]$ with respect to the probability measure $Z$.

Entropy $H$ is a concave functional of the measures:

$$ H( t X + (1-t)Y) \geq H(t X + (1-t)Y|T) = tH(X) + (1-t)H(Y) $$

here $T$ is a 0-1 Bernoulli random variable with $\mathbb{P}(T=1) = t$.

Try: $-H(Z) = \mathbb{E} [\log Z] $ we have an equality:

$$ H(Z) = -\mathbb{E} [\log \tfrac{1}{K}] - \alpha \mathbb{E} [\log X]-(1-\alpha) \mathbb{E} [\log Y] = \log K + \alpha H(X) +(1-\alpha) H(Y)$$

rearrange a bit to look like the original problem:

$$ \alpha H(X) +(1-\alpha) H(Y) = H(Z) - \log K $$

$K \leq 1$ using Hölder inequality and so $H(Z) \geq 0$ and $\log K < 0$ is negative, as in a related question on entropy inequalities

$$ H(Z) - \log K \geq H(Z) \geq K^2H(Z)$$


This is not correct as stated. In fact, $-H(Z) = \mathbb{E}_Z [\log Z]$ with respect to the probability measure $Z$.

Entropy $H$ is a concave functional of the measures:

$$ H( t X + (1-t)Y) \geq H(t X + (1-t)Y|T) = tH(X) + (1-t)H(Y) $$

here $T$ is a 0-1 Bernoulli random variable with $\mathbb{P}(T=1) = t$.


I am really going to drop it after this. The way to prove one entropy is lower than another is to use conditional entropy: $H(X) \geq H(X|Y)$

What to condition on? I use unorthodox and possibly wrong notation:

$$ tH(X) + (1-t)H(Y) = \mathbb{E}[t \log X + (1-t)\log Y | X,Y,t]$$

Given distributions $X,Y$ and Bernoulli random variable $t$ we can construct the above entropy.

$$ K^2H(Z) = \mathbb{E}[\sqrt{XY}\,\big|i]^2 \mathbb{E}[t \log X + (1-t)\log Y \,\big| Z=\tfrac{\sqrt{XY}}{K},t]$$

Where $i$ is uniform in $[1,2,\dots, n]$. It still doesn't look right.

added 328 characters in body
Source Link
john mangual
  • 22.8k
  • 4
  • 63
  • 172

Try: $-H(Z) = \mathbb{E} [\log Z] $ we have an equality:

$$ H(Z) = -\mathbb{E} [\log \tfrac{1}{K}] - \alpha \mathbb{E} [\log X]-(1-\alpha) \mathbb{E} [\log Y] = \log K + \alpha H(X) +(1-\alpha) H(Y)$$

rearrange a bit to look like the original problem:

$$ \alpha H(X) +(1-\alpha) H(Y) = H(Z) - \log K $$

$K \leq 1$ using Hölder inequality and so $H(Z) \geq 0$ and $\log K < 0$ is negative, as in a related question on entropy inequalities

$$ H(Z) - \log K \geq H(Z) \geq K^2H(Z)$$


This is not correct as stated. In fact, $-H(Z) = \mathbb{E}_Z [\log Z]$ with respect to the probability measure $Z$.

Entropy $H$ is a concave functional of the measures:

$$ H( t X + (1-t)Y) \geq H(t X + (1-t)Y|T) = tH(X) + (1-t)H(Y) $$

here $T$ is a 0-1 Bernoulli random variable with $\mathbb{P}(T=1) = t$.

Try: $-H(Z) = \mathbb{E} [\log Z] $ we have an equality:

$$ H(Z) = -\mathbb{E} [\log \tfrac{1}{K}] - \alpha \mathbb{E} [\log X]-(1-\alpha) \mathbb{E} [\log Y] = \log K + \alpha H(X) +(1-\alpha) H(Y)$$

rearrange a bit to look like the original problem:

$$ \alpha H(X) +(1-\alpha) H(Y) = H(Z) - \log K $$

$K \leq 1$ using Hölder inequality and so $H(Z) \geq 0$ and $\log K < 0$ is negative, as in a related question on entropy inequalities

$$ H(Z) - \log K \geq H(Z) \geq K^2H(Z)$$

Try: $-H(Z) = \mathbb{E} [\log Z] $ we have an equality:

$$ H(Z) = -\mathbb{E} [\log \tfrac{1}{K}] - \alpha \mathbb{E} [\log X]-(1-\alpha) \mathbb{E} [\log Y] = \log K + \alpha H(X) +(1-\alpha) H(Y)$$

rearrange a bit to look like the original problem:

$$ \alpha H(X) +(1-\alpha) H(Y) = H(Z) - \log K $$

$K \leq 1$ using Hölder inequality and so $H(Z) \geq 0$ and $\log K < 0$ is negative, as in a related question on entropy inequalities

$$ H(Z) - \log K \geq H(Z) \geq K^2H(Z)$$


This is not correct as stated. In fact, $-H(Z) = \mathbb{E}_Z [\log Z]$ with respect to the probability measure $Z$.

Entropy $H$ is a concave functional of the measures:

$$ H( t X + (1-t)Y) \geq H(t X + (1-t)Y|T) = tH(X) + (1-t)H(Y) $$

here $T$ is a 0-1 Bernoulli random variable with $\mathbb{P}(T=1) = t$.

simplification
Source Link
john mangual
  • 22.8k
  • 4
  • 63
  • 172
Loading
Source Link
john mangual
  • 22.8k
  • 4
  • 63
  • 172
Loading