21
$\begingroup$

Cramer's decomposition theorem states that if $X$ and $Y$ are independent real random variables and $X+Y$ has normal distribution, then both $X$ and $Y$ are normally distributed. I've seen a few proofs of this result which involve grunt work in the realm of complex analysis. I was wondering if there is an intuitive proof of this result. I would like to see a proof that exploits the unique nature of the normal distribution. For example, can one derive the result from the fact that on the real line, the normal distribution maximizes entropy for a given mean and standard deviation?

Edit: The complex analysis proof I'm thinking of uses the fact that if $E[\exp(\alpha X^2)]<\infty$ for some $\alpha>0$ and the analytic continuation of the characteristic function of $X$ is nonzero, then $X$ is normally distributed.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
7
$\begingroup$

For example, can one derive the result from the fact that on the real line, the normal distribution maximizes entropy for a given mean and standard deviation?

That sounds like a workable approach. Here's an intuitive proof:

Visualize the convolution of two functions as the smoothing of one function by the other one's shape. Smoothing can only spread out probability mass and not concentrate it, so entropy cannot decrease in the process. If one function is kept fixed while the entropy of the other is increased, the entropy of the convolution also increases.

Now suppose $X$ and $Y$ are independent random variables such that $X + Y$ has maximum entropy among those random variables with its mean and variance, but $X$ does not have maximum entropy in its mean and variance class. Let $X'$ be a random variable with the same mean and variance as $X$ but greater entropy. Then $X' + Y$ has the same mean and variance as $X + Y$, and by the entropy convolution inequality in the previous paragraph $X' + Y$ has greater entropy than $X + Y$. But this is a contradiction, so $X$ and by symmetry $Y$ must have maximum entropy.

Improve this answer
$\endgroup$
4
  • 1
    $\begingroup$ I guess you should make the above argument more qualitative to work (you need to compare entropies of $X+Y$ and $X'+Y$, not only of $X$ and $X+Y$), but this seems interesting. $\endgroup$
    – Benoît Kloeckner
    Commented Sep 21, 2010 at 8:57
  • $\begingroup$ Ah, I see what you mean now. I need to strengthen the inequality I outlined in the first paragraph. $\endgroup$
    – Per Vognsen
    Commented Sep 21, 2010 at 9:16
  • 1
    $\begingroup$ "If one function is kept fixed while the entropy of the other is increased, the entropy of the convolution also increases". This would imply that convolving one function with two functions of the same entropy results in functions of the same entropy and, in one more step, that the entropy of the convolution depends only on the entropies of the functions convolved. Do you really believe that? $\endgroup$
    – fedja
    Commented Sep 21, 2010 at 21:52
  • 1
    $\begingroup$ No, you're of course right, that doesn't make sense. Do you see a simple way of repairing the argument? The whole idea is based on the picture I have in my head of what happens when $X$ has less than maximum entropy. Then it can be smoothed further towards a Gaussian which in turn further smoothes out $X + Y$. $\endgroup$
    – Per Vognsen
    Commented Sep 22, 2010 at 0:47

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .