MathOverflow is a question and answer site for professional mathematicians. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

What is the entropy of a normal distribution with mean 0 and variance \sigma? Thanks!

share|cite|improve this question

closed as too localized by Andrey Rekalo, Yemon Choi, Qiaochu Yuan, Robin Chapman, Harald Hanche-Olsen Jul 9 '10 at 23:39

This question is unlikely to help any future visitors; it is only relevant to a small geographic area, a specific moment in time, or an extraordinarily narrow situation that is not generally applicable to the worldwide audience of the internet. For help making this question more broadly applicable, visit the help center.If this question can be reworded to fit the rules in the help center, please edit the question.

Typing the question into Google seems to give plenty of leads; also, Wikipedia looks to have answers that seem a bout right. Voting to close. – Yemon Choi Jul 9 '10 at 20:42
Fine work on the World Cup.… – Will Jagy Jul 9 '10 at 21:02
up vote 4 down vote accepted

I found here that "the negative differential entropy of the normal distribution" (which may not be what you are asking for?) is: $$-\frac{1}{2} [ \log (2 \pi \sigma^2 ) + 1 ] ,$$ independent of $\mu$.

share|cite|improve this answer
That's what I got too when i manually derived the formula of E[log(p(x))]. however, the wikipedia formula for entropy of a normal distribution differs by a term of 1/2... Would you have some insight on why that is? Thanks, -Octi – octopus Jul 9 '10 at 20:51
Octopus: Wikipedia is a big site. Care to tell us exactly what page you are looking at with a link? At the table on the right at the top lists the entropy as $(1/2)\log(2\pi{e}\sigma^2)$, which is the same thing as $(1/2)log(2\pi\sigma^2) + 1/2$ since $\log(e) = 1$. So this is the same formula Joseph gives up to an overall minus sign (I wouldn't use "negative" entropy; the standard entropy formula should have an overall minus sign in it which cancels out that negative sign.) – KConrad Jul 9 '10 at 21:12
@KConrad: Thanks for responding while I was out of touch! Yes, I think the discrepancy can be resolved by manipulations of $\log(e)$ and minus signs. – Joseph O'Rourke Jul 9 '10 at 23:38
Ok I apologize I didn't see that Joseph's formula didn't have the e in the log argument. Thanks KConrad for pointing that out! Now this seems to be consistent everywhere! – octopus Jul 12 '10 at 1:40

Not the answer you're looking for? Browse other questions tagged or ask your own question.