What is the entropy of a normal distribution with mean 0 and variance \sigma? Thanks!
2
-
4$\begingroup$ Typing the question into Google seems to give plenty of leads; also, Wikipedia looks to have answers that seem a bout right. Voting to close. $\endgroup$– Yemon ChoiCommented Jul 9, 2010 at 20:42
-
14$\begingroup$ Fine work on the World Cup. www1.voanews.com/english/news/sports/… $\endgroup$– Will JagyCommented Jul 9, 2010 at 21:02
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
4
I found here that "the negative differential entropy of the normal distribution" (which may not be what you are asking for?) is: $$-\frac{1}{2} [ \log (2 \pi \sigma^2 ) + 1 ] ,$$ independent of $\mu$.
answered Jul 9, 2010 at 20:38
-
$\begingroup$ That's what I got too when i manually derived the formula of E[log(p(x))]. however, the wikipedia formula for entropy of a normal distribution differs by a term of 1/2... Would you have some insight on why that is? Thanks, -Octi $\endgroup$– octopusCommented Jul 9, 2010 at 20:51
-
2$\begingroup$ Octopus: Wikipedia is a big site. Care to tell us exactly what page you are looking at with a link? At en.wikipedia.org/wiki/Normal_distribution the table on the right at the top lists the entropy as $(1/2)\log(2\pi{e}\sigma^2)$, which is the same thing as $(1/2)log(2\pi\sigma^2) + 1/2$ since $\log(e) = 1$. So this is the same formula Joseph gives up to an overall minus sign (I wouldn't use "negative" entropy; the standard entropy formula should have an overall minus sign in it which cancels out that negative sign.) $\endgroup$– KConradCommented Jul 9, 2010 at 21:12
-
$\begingroup$ @KConrad: Thanks for responding while I was out of touch! Yes, I think the discrepancy can be resolved by manipulations of $\log(e)$ and minus signs. $\endgroup$ Commented Jul 9, 2010 at 23:38
-
$\begingroup$ Ok I apologize I didn't see that Joseph's formula didn't have the e in the log argument. Thanks KConrad for pointing that out! Now this seems to be consistent everywhere! $\endgroup$– octopusCommented Jul 12, 2010 at 1:40