Skip to main content
2 of 5
added 5 characters in body
Michael Hardy
  • 1
  • 12
  • 85
  • 126

If you apply Bessel's correction --- dividing by $5-1$ rather than by $5$ when you have $5$ numbers --- then some of the otherwise right things stated in some of the answers are wrong. Bessel's correction is intended to be used only when the variance one is computing is based on a sample to be used to estimate the variance of the whole population.

I won't be surprised if nobody used the variance and standard deviation before Abraham de Moivre did so in the 18th century. De Moivre considered this question: If you toss a fair coin $1800$ times, what is the probability that the number of heads is in some specified range? You have a binomial distribution, and computing its exact values was not feasible. De Moivre approximated the distribution of the number of heads with a normal distribution with the same mean and the same standard deviation. In so doing, he was the first to introduce the normal distribution, and the first to prove a special case of the central limit theorem. The normal distribution with mean $0$ and variance $1$ is $$ \varphi(x)\,dx=\frac 1 {\sqrt{2\pi}} e^{-x^2/2}\,dx $$ and with mean $\mu$ and variance $\sigma^2$ it is $$ \varphi\left(\frac{x-\mu}\sigma\right)\, \frac{dx}\sigma. $$ It's easy to find the mean and standard deviation for the number of heads when one fair coin is tossed: they're both $1/2$. How do you do it for the sum of $1800$ independent copies of that random variable? De Moivre found that the mean-square deviation is additive: for independent random variables $X_1,\ldots,X_{1800}$ one has $\operatorname{var}(X_1+\cdots+X_{1800})=\operatorname{var}(X_1)+\cdots+\operatorname{var}(X_{1800})$. You cannot do that with mean absolute deviation. If I'm recalling some details correctly, he published these findings in a paper in Latin while he lived in France, and at that time he gave the normal distribution as $$ C e^{-x^2/2}\,dx $$ where he could find $C$ only numerically. Later he went to England to escape the persecution of Protestants and met James Stirling, who showed that $C=1/\sqrt{2\pi}$. De Moivre wrote a book in English called The Doctrine of Chances, which I think was 18th-century English for the theory of probability. Some have speculated that the Reverend Thomas Bayes may have studied under him, but I don't know that that's gone beyond speculation.

(If you want to know the probability that the number of heads is $\ge894$, note that that's the same as "$>893$", and find the probability that the normally distributed random variable with the same mean and variance is $>893.5$. That is a "continuity correction" and works surprisingly well even for fairly small samples.)

On to Bessel's correction: When does one use $$ \frac{(x_1-\bar x)^2+\cdots+(x_n-\bar x)^2}{n-1}, $$ where $\bar x=(x_1+\cdots+x_n)/n$, with $n-1$ rather than $n$ in the denominator? As you can see from simple examples, that will not serve de Moivre's purpose described above: it's not additive.

If $X_1,\ldots,X_n$ are an independent sample from a population with mean $\mu$ and variance $\sigma^2$, then the expected value of $$ \frac{(X_1-\mu)^2+\cdots+(X_n-\mu)^2} n \tag 1 $$ is $\sigma^2$. But if one has only the sample and not the whole population, one doesn't know $\mu$ and one can use the sample mean $\bar X$ as an estimate of $\mu$. But the expected value of $$ \frac{(X_1-\bar X)^2+\cdots+(X_n-\bar X)^2} n $$ is smaller than the expected value of $(1)$. Specifically, a bit of algebra shows that $$ \sum_{i=1}^n (X_i-\mu)^2 = \left( \sum_{i=1}^n (X_i-\bar X)^2 \right) + n(\bar X-\mu)^2, \tag 2 $$ and since the expectation of the last term is $\sigma^2$, that of the first term on the right in $(2)$ must be $(n-1)\sigma^2$. Thus Bessel's correction gives an unbiased estimate of the population variance $\sigma^2$. (But its square root does not give an unbiased estimate of the population standard deviation. And unbiasedness is at best somewhat overrated anyway, and in some cases is a very very bad thing (I had a paper in the American Mathematical Monthly a few years ago demonstrating how bad it can sometimes be).

Michael Hardy
  • 1
  • 12
  • 85
  • 126