Skip to main content
4 of 6
Tried to improve clarity; added 11 characters in body
Per Vognsen
  • 2.1k
  • 19
  • 24

For example, can one derive the result from the fact that on the real line, the normal distribution maximizes entropy for a given mean and standard deviation?

That sounds like a workable approach. Here's an intuitive proof:

The entropy of a convolution is greater than the entropy of the convolved functions. Why? Visualize the convolution of two functions as the smoothing of one function by the other one's shape. Smoothing can only spread out probability mass and not concentrate it, so entropy cannot decrease in the process.

Now suppose $X$ and $Y$ are independent random variables such that $X + Y$ has maximum entropy among those random variables with its mean and variance, but $X$ does not have maximum entropy in its mean and variance class. Let $X'$ be a random variable with the same mean and variance as $X$ but greater entropy. Then $X' + Y$ has the same mean and variance as $X + Y$. By the entropy convolution inequality in the previous paragraph, $X' + Y$ has greater entropy than $X + Y$. But this is a contradiction, so $X$ and by symmetry $Y$ must have maximum entropy.

Per Vognsen
  • 2.1k
  • 19
  • 24