Take the 2-minute tour ×
MathOverflow is a question and answer site for professional mathematicians. It's 100% free, no registration required.

When we want to find the standard deviation of $\{1,2,2,3,5\}$ we do

$$\sigma = \sqrt{ {1 \over 5-1} \left( (1-2.6)^2 + (2-2.6)^2 + (3-2.6)^2 + (5 - 2.6^2) \right) } \approx 1.52$$.

Why do we need to square and then square-root the numbers?

share|improve this question
1  
+1, excellent question, though I think the title can be improved. I think you selected the wrong answer though — there are many better reasons for the deviation formula than convenience. –  Ilya Nikokoshev Oct 22 '09 at 18:07
1  
I hope that (over four years later) someone will return to this classical question. There seems to be something special about the number $p=2$ in $L_p$ norms. What is it? Flatness? Something exotic? (whatever makes Gleason's theorem work?) It seems to me that whatever makes $2$ special is independent of the CLT. It would be disappointing if the Gaussian assumption is the final reason for $p=2$. –  isomorphismes Jan 14 at 16:36

9 Answers 9

up vote -3 down vote accepted

One answer is mathematical convenience. The theory is much simpler using powers of two rather than other powers.

There are justifications. Squaring makes small numbers smaller and makes big numbers bigger. You could argue that a useful measure of dispersion should be forgiving of small errors but weigh larger errors more heavily, so it makes sense to square the deviations from the mean.

share|improve this answer
1  
Sorry for -1, but you miss the key point, see my answer. –  Ilya Nikokoshev Oct 22 '09 at 18:04

Short answer: You could argue that the most natural thing to do when defining a "standard deviation-type" quantity is to use an absolute value: E( |X| ), but its really annoying to deal with absolute values under the expectation, so we use the next best thing: E( X^2 )^(1/2). You still get something positive and its easier to deal with the square inside. We take a square root at the end to get something with the same "units" as X.

Long answer: It's often helpful to think of random variables as living in the function space L^2(Omega), and in this setting, this computation gives the L^2 norm of the centered random variable X - EX. Also, with this perspective, the covariance defines is a inner product.

share|improve this answer
    
Sorry, I think I was too rush in downvoting. The L^2 observation makes a lot of sense and is quite deep. –  Ilya Nikokoshev Oct 24 '09 at 20:31
6  
What definitions are you using for "short" and "long"? –  Cap Khoury Jul 30 '10 at 1:26
    
But why is $L^2$ cooler than $L^4$ or some other absolute value/function space? –  isomorphismes Jan 14 at 16:17
    
@isomorphismes It has more random variables with finite norm. And it implies that two random variables with finite variance have also a finite covariance. –  Stephan Sturm Jan 14 at 17:55

With standard deviation as defined, you get cool results like Chebyshev's Theorem: for any distribution and k>1, at most 1/k^2 of the data fall outside of k standard deviations from the mean. So, for example, for any distribution at most a quarter of the data lies farther then two standard deviations from the mean, and at most 12% lie further than three standard deviations.

This and other theoretical advantages come from the long answer that Mark4483 gave. These things are important for developing inference models.

share|improve this answer
1  
If we define the standard deviation with absolute value instead of squares, we would still have Chebyshev's theorem except with 1/k in place of 1/k^2 –  Yoo Nov 5 '09 at 7:57

One answer I've heard is that you want the notion of standard deviation to 1) apply to points in Euclidean space, and 2) to be invariant under rotation. You don't get the second property unless you square the distances.

share|improve this answer
1  
What does "rotation" mean in statistics? It's clear in physics why we should prize rotation-invariance, but if our three measurements of a person are the length of their surname + their height + their wealth, then I would only compute SD 1-dimensionally. –  isomorphismes Jan 14 at 16:22
    
I guess PCA involves rotation... –  isomorphismes Jan 16 at 4:32

Intro by Reid Barton

I think the answer should involve the additivity of variance for independent variables and the central limit theorem. Maybe someone can flesh this out.

Answer

Indeed, the variance has the additivity property: if r_1 is a random variable with mean m_1 and variance d_1 and r_2 is a random variable with mean m_2 and variance d_2 and these two variables are independent then the new random variable r = r_1+r_2 has the mean m_1+m_2 and variance d_1+d_2.

Moreover, suppose we sum a large number N of independent copies of our random variable r with mean m and variance d. Under mild assumptions, the central limit says the distribution will approach a normal distribution, which by the above has mean Nm and variance Nd. Observe that a normal distribution is completely determined by its mean and variance. We conclude that the only parameters of a distribution that we can observe from the sum of many independent copies of the distribution are the mean and variance.

Now that we established how good it is to square numbers, to get variance, the standard deviation has a very easy explanation: it's the only way to get back from variance to something with the dimension of our original set. That is, suppose you numbers are some lengths written in meters. Since the variance is meters squared, you have to take the square root to get something that can be compared with the original set.

Now, honestly, this not the only way, since you could also, e.g., multiply it by 2. That's why it's called standard deviation — to show that among different numerical constants we've chosen a specific one.

share|improve this answer
2  
Excellent use of community wiki! –  Anton Geraschenko Oct 19 '09 at 7:00
    
Ilya seems to have confused variance and standard deviation. The variance of the sum of independent random variables is the sum of the variances: Var(X+Y)=Var(X)+Var(Y) so StdDev(X+Y)=sqrt(Var(X)+Var(Y)), which in general is not equal to StdDev(X)+StdDev(Y) –  Robert Parviainen Oct 22 '09 at 18:22
    
Fixed, and included the relation to the central limit theorem. –  Reid Barton Oct 22 '09 at 18:41
    
Ah, sorry, I also included a longer but essentially the same paragraph as yours. –  Ilya Nikokoshev Oct 22 '09 at 18:44
2  
I think you guys are dismissing the L^2 interpretation of this question. Variance is important, but it is just a special case of covariance, which is where the real action is going on. All the nice properties of variance follow from the fact that covariance is more or less an inner product, and independent random variables are orthogonal with respect to that inner product. –  Dinakar Muthiah Oct 24 '09 at 19:27

The question seems to be asked from a statistics point of view.

In statistics, (sample) standard deviation is used as a measure of dispersion in data. It does have many nice properties, but as others have said, why we choose it is often due to convenience.

The convenience stems from the fact that we often want to minimize dispersion. With standard deviation, such minimization problems tend to have explicit solutions, while for other dispersion measures numeric methods has to be used to a much higher degree. Also, explicit solutions simplify statistical theory a lot.

It does have its drawbacks however, and is not the only dispersion measure used in practice. A big drawback is that it is sensitive to outliers in data. If to the data set {1,2,2,3,5}, the point 100 is added, the standard deviation changes a lot. It is not the best measure for highly skewed data, or data from distributions with heavy tails.

One alternative is the interquartile range, the distance between the 3rd and 1st quartiles, and mean absolute distance to the median is another.

See the wikipedia article on Robust Statistics for more information.

share|improve this answer

The answers here that stress that the answers here that stress convenience are missing the crucial point are missing a crucial point. There are at least two ways to approach variance (which are sort of dual to one another):

(1) I need to measure the spread of a distribution of values. What measure should I use? [Possible answer: variance.]

-or-

(2) If I define V[X] = E[(X-E[X])^2], then V[X] has many nice properties and seems to relate well to other parts of the theory and even other parts of mathematics. Obviously, it's something pretty important theoretically. What are its practical uses? [Possible answer: as a measure of spread.]

If the only thing you care about is measuring spread, then convenience may be the only reason for you to use the variance, if you use it at all. I don't think anybody can seriously claim that among all measures of spread, the variance is absolutely the best-quality measure in all situations. Yes, the variance is additive and allows you to formulate the central limit theorem, and properties like that are certainly very nice to have, both in theory and in practice, but they don't make it a better measure of spread. So from this perspective, "convenience" often is the right answer.

On the other hand, as a theorist, you would probably want to develop the theory along the most fruitful route, so you would be stupid to ignore the variance in any case. Its usefulness as a measure of spread then is less important, and its overall properties are rather more important. From this perspective, "convenience" isn't really the right answer, since it doesn't convey the intrinsic value it has by virtue of the excellent theory surrounding it.

share|improve this answer
1  
Interesting (and +1) –  Ilya Nikokoshev Oct 24 '09 at 20:27
2  
But no, I think you're stepping on a bit confusing path. Yes, if you just define the variance, it is convenient. But why is it convenient? You should say something about the reason, otherwise you're just changing a mystery as for why to use variance into a mystery as for why variance is convenient. –  Ilya Nikokoshev Oct 24 '09 at 20:30
    
Though I agree may be I should have been more respectful to L^2 argument. –  Ilya Nikokoshev Oct 24 '09 at 20:31

Population variance, E([X-E(X)]^2), can quite easily be estimated in an unbiased manner by the sample variance, (n-1)^{-1}\sum (X_i-\bar{X})^2, where the sum is from i=1 to i=n. "Unbiased" meaning that if the X_i are i.i.d. copies of X, then the expectation of the sample variance (by a straightforward direct computation) is the population variance.

As far as I know, you can't do anything at all for absolute values. For even powers larger than 2, a more complicated unbiased estimator could be constructed. Two is the simplest case, and plenty of interest happens - the central limit theorem, for starters. I'm not sure to what extent the L^2 theory generalizes to L^2k theory, but to my knowledge no compelling reason to consider it exists. Perhaps with higher k, the large deviations are weighted even more heavily, and particular applications may benefit from their study. However, the usual variance is a good starting point. There are conditions under which you can recover a random variable X if you know all of its moments, E(X^p) for p = 1, 2, 3, .... From this perspective, one might consider the variance, E(X^2)-E(X)^2, a function of the first two moments, for the same reason that one may look at a second-degree Taylor approximation of sin - it's the simplest, and you can look at other terms if you need to know more.

share|improve this answer

I think that is quite obvious.

Firstly, the sampling standard deviation measures the dispersion of the data in the samples.

You square the difference is to reflect how much the actually data is away from the sample mean. If you do not square them, some differences may cancel out each other e.g. -1, +1, in which case the dispersion cannot be reflected.

You square root it is for matching the dimension

share|improve this answer
3  
There are more direct ways to deal with canceling, such as averaging the magnitude of the difference. –  Hurkyl Sep 10 '11 at 15:46

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.