5
$\begingroup$

What is the entropy of a normal distribution with mean 0 and variance \sigma? Thanks!

$\endgroup$
2
  • 4
    $\begingroup$ Typing the question into Google seems to give plenty of leads; also, Wikipedia looks to have answers that seem a bout right. Voting to close. $\endgroup$
    – Yemon Choi
    Jul 9, 2010 at 20:42
  • 14
    $\begingroup$ Fine work on the World Cup. www1.voanews.com/english/news/sports/… $\endgroup$
    – Will Jagy
    Jul 9, 2010 at 21:02

1 Answer 1

6
$\begingroup$

I found here that "the negative differential entropy of the normal distribution" (which may not be what you are asking for?) is: $$-\frac{1}{2} [ \log (2 \pi \sigma^2 ) + 1 ] ,$$ independent of $\mu$.

$\endgroup$
4
  • $\begingroup$ That's what I got too when i manually derived the formula of E[log(p(x))]. however, the wikipedia formula for entropy of a normal distribution differs by a term of 1/2... Would you have some insight on why that is? Thanks, -Octi $\endgroup$
    – octopus
    Jul 9, 2010 at 20:51
  • 2
    $\begingroup$ Octopus: Wikipedia is a big site. Care to tell us exactly what page you are looking at with a link? At en.wikipedia.org/wiki/Normal_distribution the table on the right at the top lists the entropy as $(1/2)\log(2\pi{e}\sigma^2)$, which is the same thing as $(1/2)log(2\pi\sigma^2) + 1/2$ since $\log(e) = 1$. So this is the same formula Joseph gives up to an overall minus sign (I wouldn't use "negative" entropy; the standard entropy formula should have an overall minus sign in it which cancels out that negative sign.) $\endgroup$
    – KConrad
    Jul 9, 2010 at 21:12
  • $\begingroup$ @KConrad: Thanks for responding while I was out of touch! Yes, I think the discrepancy can be resolved by manipulations of $\log(e)$ and minus signs. $\endgroup$ Jul 9, 2010 at 23:38
  • $\begingroup$ Ok I apologize I didn't see that Joseph's formula didn't have the e in the log argument. Thanks KConrad for pointing that out! Now this seems to be consistent everywhere! $\endgroup$
    – octopus
    Jul 12, 2010 at 1:40

Not the answer you're looking for? Browse other questions tagged or ask your own question.