Return to Answer

For example, can one derive the result from the fact that on the real line, the normal distribution maximizes entropy for a given mean and standard deviation?

That sounds like a workable approach. Here's an intuitive proof:

The entropy of a convolution is greater than the entropy of the convolved functions. Why? Visualize the convolution of two functions as the smoothing of one function by the other one's shape. Smoothing can only spread out probability mass and not concentrate it, so entropy cannot decrease in the process.

Now suppose $X$ and $Y$ are independent random variables such that $X + Y$ has maximum entropy among those random variables with its mean and variance, but $X$ does not have maximum entropy in its mean and variance class. Let $X'$ be a random variable with the same mean and variance as $X$ but greater entropy. Then $X' + Y$ has the same mean and variance as $X + Y$, and by the entropy convolution inequality in the previous paragraph $X' + Y$ has greater entropy than $X + Y$. But this is a contradiction, so $X$ and by symmetry $Y$ must have maximum entropy.

For example, can one derive the result from the fact that on the real line, the normal distribution maximizes entropy for a given mean and standard deviation?

That sounds like a workable approach. Here's an intuitive proof:

Visualize the convolution of two functions as the smoothing of one function by the other one's shape. Smoothing can only spread out probability mass and not concentrate it, so entropy cannot decrease in the process. If one function is kept fixed while the entropy of the other is increased, the entropy of the convolution also increases.

Now suppose $X$ and $Y$ are independent random variables such that $X + Y$ has maximum entropy among those random variables with its mean and variance, but $X$ does not have maximum entropy in its mean and variance class. Let $X'$ be a random variable with the same mean and variance as $X$ but greater entropy. Then $X' + Y$ has the same mean and variance as $X + Y$, and by the entropy convolution inequality in the previous paragraph $X' + Y$ has greater entropy than $X + Y$. But this is a contradiction, so $X$ and by symmetry $Y$ must have maximum entropy.

For example, can one derive the result from the fact that on the real line, the normal distribution maximizes entropy for a given mean and standard deviation?

That sounds like a workable approach. Here's an intuitive proof:

The entropy of a convolution is greater than the entropy of the convolved functions. Why? Visualize the convolution of two functions as the smoothing of one function by the other one's shape. Smoothing can only spread out probability mass and not concentrate it, so entropy cannot decrease in the process.

Now suppose $X$ and $Y$ are independent random variables such that $X + Y$ has maximum entropy among those random variables with its mean and variance, but $X$ does not have maximum entropy in its mean and variance class. Let $X'$ be a random variable with the same mean and variance as $X$ but greater entropy. Then $X' + Y$ has the same mean and variance as $X + Y$. By the entropy convolution inequality in the previous paragraph, $X' + Y$ has greater entropy than $X + Y$. But this is a contradiction, so $X$ and by symmetry $Y$ must have maximum entropy.

For example, can one derive the result from the fact that on the real line, the normal distribution maximizes entropy for a given mean and standard deviation?

That sounds like a workable approach. Here's an intuitive proof:

The entropy of a convolution is greater than the entropy of the convolved functions. Why? Visualize the convolution of two functions as the smoothing of one function by the other one's shape. Smoothing can only spread out probability mass and not concentrate it, so entropy cannot decrease in the process.

Now suppose $X$ and $Y$ are independent random variables such that $X + Y$ has maximum entropy among those random variables with its mean and variance, but $X$ does not have maximum entropy in its mean and variance class. Let $X'$ be a random variable with the same mean and variance as $X$ but greater entropy. Then $X' + Y$ has the same mean and variance as $X + Y$, and by the entropy convolution inequality in the previous paragraph $X' + Y$ has greater entropy than $X + Y$. But this is a contradiction, so $X$ and by symmetry $Y$ must have maximum entropy.

For example, can one derive the result from the fact that on the real line, the normal distribution maximizes entropy for a given mean and standard deviation?

That sounds like a workable approach. Here's an intuitive proof:

The entropy of a convolution is greater than the entropy of the convolved functions. Why? Visualize the convolution of two functions as the smoothing of one function by the other one's shape. Smoothing can only spread out probability mass and not concentrate it, so entropy cannot decrease in the process.

Now suppose $X$ and $Y$ are independent random variables such that $X + Y$ has maximum entropy among those random variables with its mean and variance, but $X$ does not have maximum entropy in its mean and variance class. If $X$ is replaced with a random variable of identical mean and variance but greater entropy, you get an $X + Y$ with the same mean and variance as the old $X + Y$. However, by the entropy convolution inequality in the previous paragraph, the new $X + Y$ must have greater entropy than the old one. But this is a contradiction, so $X$ and by symmetry $Y$ must have maximum entropy.

For example, can one derive the result from the fact that on the real line, the normal distribution maximizes entropy for a given mean and standard deviation?

That sounds like a workable approach. Here's an intuitive proof:

The entropy of a convolution is greater than the entropy of the convolved functions. Why? Visualize the convolution of two functions as the smoothing of one function by the other one's shape. Smoothing can only spread out probability mass and not concentrate it, so entropy cannot decrease in the process.

Now suppose $X$ and $Y$ are independent random variables such that $X + Y$ has maximum entropy among those random variables with its mean and variance, but $X$ does not have maximum entropy in its mean and variance class. Let $X'$ be a random variable with the same mean and variance as $X$ but greater entropy. Then $X' + Y$ has the same mean and variance as $X + Y$. By the entropy convolution inequality in the previous paragraph, $X' + Y$ has greater entropy than $X + Y$. But this is a contradiction, so $X$ and by symmetry $Y$ must have maximum entropy.