MathOverflow will be down for maintenance for approximately 3 hours, starting Monday evening (06/24/2013) at approximately 9:00 PM Eastern time (UTC-4).
1

What is the entropy of a normal distribution with mean 0 and variance \sigma? Thanks!

flag
4 
 Typing the question into Google seems to give plenty of leads; also, Wikipedia looks to have answers that seem a bout right. Voting to close. – Yemon Choi Jul 9 2010 at 20:42
12 
 Fine work on the World Cup. www1.voanews.com/english/news/sports/… – Will Jagy Jul 9 2010 at 21:02

closed as too localized by Andrey Rekalo, Yemon Choi, Qiaochu Yuan, Robin Chapman, Harald Hanche-Olsen Jul 9 2010 at 23:39

1 Answer

3

I found here that "the negative differential entropy of the normal distribution" (which may not be what you are asking for?) is: $$-\frac{1}{2} [ \log (2 \pi \sigma^2 ) + 1 ] ,$$ independent of $\mu$.

link|flag
That's what I got too when i manually derived the formula of E[log(p(x))]. however, the wikipedia formula for entropy of a normal distribution differs by a term of 1/2... Would you have some insight on why that is? Thanks, -Octi – octopus Jul 9 2010 at 20:51
2 
 Octopus: Wikipedia is a big site. Care to tell us exactly what page you are looking at with a link? At en.wikipedia.org/wiki/Normal_distribution the table on the right at the top lists the entropy as $(1/2)\log(2\pi{e}\sigma^2)$, which is the same thing as $(1/2)log(2\pi\sigma^2) + 1/2$ since $\log(e) = 1$. So this is the same formula Joseph gives up to an overall minus sign (I wouldn't use "negative" entropy; the standard entropy formula should have an overall minus sign in it which cancels out that negative sign.) – KConrad Jul 9 2010 at 21:12
@KConrad: Thanks for responding while I was out of touch! Yes, I think the discrepancy can be resolved by manipulations of $\log(e)$ and minus signs. – Joseph O'Rourke Jul 9 2010 at 23:38
Ok I apologize I didn't see that Joseph's formula didn't have the e in the log argument. Thanks KConrad for pointing that out! Now this seems to be consistent everywhere! – octopus Jul 12 2010 at 1:40

Not the answer you're looking for? Browse other questions tagged or ask your own question.