why is it so cool to square numbers? (in terms of finding the standard deviation) - MathOverflow

why is it so cool to square numbers? (in terms of finding the standard deviation)

2009-10-18T16:59:45Z

So when we want to find the standard deviation of {1,2,2,3,5} we do

http://controls.engin.umich.edu/wiki/images/math/0/0/6/00665fe48315006861c0755b6081bc49.png

why do we need to square and then square root the numbers?

Answer by Mark4483 for why is it so cool to square numbers? (in terms of finding the standard deviation)

2009-10-18T17:17:17Z

Short answer: You could argue that the most natural thing to do when defining a "standard deviation-type" quantity is to use an absolute value: E( |X| ), but its really annoying to deal with absolute values under the expectation, so we use the next best thing: E( X^2 )^(1/2). You still get something positive and its easier to deal with the square inside. We take a square root at the end to get something with the same "units" as X.

Long answer: It's often helpful to think of random variables as living in the function space L^2(Omega), and in this setting, this computation gives the L^2 norm of the centered random variable X - EX. Also, with this perspective, the covariance defines is a inner product.

Answer by John D. Cook for why is it so cool to square numbers? (in terms of finding the standard deviation)

2009-10-18T18:29:09Z

One answer is mathematical convenience. The theory is much simpler using powers of two rather than other powers.

There are justifications. Squaring makes small numbers smaller and makes big numbers bigger. You could argue that a useful measure of dispersion should be forgiving of small errors but weigh larger errors more heavily, so it makes sense to square the deviations from the mean.

Answer by Anna Varvak for why is it so cool to square numbers? (in terms of finding the standard deviation)

2009-10-18T22:02:09Z

With standard deviation as defined, you get cool results like Chebyshev's Theorem: for any distribution and k>1, at most 1/k^2 of the data fall outside of k standard deviations from the mean. So, for example, for any distribution at most a quarter of the data lies farther then two standard deviations from the mean, and at most 12% lie further than three standard deviations.

This and other theoretical advantages come from the long answer that Mark4483 gave. These things are important for developing inference models.

Answer by Qiaochu Yuan for why is it so cool to square numbers? (in terms of finding the standard deviation)

2009-10-18T22:06:51Z

One answer I've heard is that you want the notion of standard deviation to 1) apply to points in Euclidean space, and 2) to be invariant under rotation. You don't get the second property unless you square the distances.

Answer by Reid Barton for why is it so cool to square numbers? (in terms of finding the standard deviation)

2009-10-18T22:17:11Z

Intro by Reid Barton

I think the answer should involve the additivity of variance for independent variables and the central limit theorem. Maybe someone can flesh this out.

Answer

Indeed, the variance has the additivity property: if r_1 is a random variable with mean m_1 and variance d_1 and r_2 is a random variable with mean m_2 and variance d_2 and these two variables are independent then the new random variable r = r_1+r_2 has the mean m_1+m_2 and variance d_1+d_2.

Moreover, suppose we sum a large number N of independent copies of our random variable r with mean m and variance d. Under mild assumptions, the central limit says the distribution will approach a normal distribution, which by the above has mean Nm and variance Nd. Observe that a normal distribution is completely determined by its mean and variance. We conclude that the only parameters of a distribution that we can observe from the sum of many independent copies of the distribution are the mean and variance.

Now that we established how good it is to square numbers, to get variance, the standard deviation has a very easy explanation: it's the only way to get back from variance to something with the dimension of our original set. That is, suppose you numbers are some lengths written in meters. Since the variance is meters squared, you have to take the square root to get something that can be compared with the original set.

Now, honestly, this not the only way, since you could also, e.g., multiply it by 2. That's why it's called standard deviation — to show that among different numerical constants we've chosen a specific one.

Answer by Robert Parviainen for why is it so cool to square numbers? (in terms of finding the standard deviation)

2009-10-22T19:02:02Z

The question seems to be asked from a statistics point of view.

In statistics, (sample) standard deviation is used as a measure of dispersion in data. It does have many nice properties, but as others have said, why we choose it is often due to convenience.

The convenience stems from the fact that we often want to minimize dispersion. With standard deviation, such minimization problems tend to have explicit solutions, while for other dispersion measures numeric methods has to be used to a much higher degree. Also, explicit solutions simplify statistical theory a lot.

It does have its drawbacks however, and is not the only dispersion measure used in practice. A big drawback is that it is sensitive to outliers in data. If to the data set {1,2,2,3,5}, the point 100 is added, the standard deviation changes a lot. It is not the best measure for highly skewed data, or data from distributions with heavy tails.

One alternative is the interquartile range, the distance between the 3rd and 1st quartiles, and mean absolute distance to the median is another.

See the wikipedia article on Robust Statistics for more information.

Answer by Darsh Ranjan for why is it so cool to square numbers? (in terms of finding the standard deviation)

2009-10-24T12:10:36Z

The answers here that stress that the answers here that stress convenience are missing the crucial point are missing a crucial point. There are at least two ways to approach variance (which are sort of dual to one another):

(1) I need to measure the spread of a distribution of values. What measure should I use? [Possible answer: variance.]

-or-

(2) If I define V[X] = E[(X-E[X])^2], then V[X] has many nice properties and seems to relate well to other parts of the theory and even other parts of mathematics. Obviously, it's something pretty important theoretically. What are its practical uses? [Possible answer: as a measure of spread.]

If the only thing you care about is measuring spread, then convenience may be the only reason for you to use the variance, if you use it at all. I don't think anybody can seriously claim that among all measures of spread, the variance is absolutely the best-quality measure in all situations. Yes, the variance is additive and allows you to formulate the central limit theorem, and properties like that are certainly very nice to have, both in theory and in practice, but they don't make it a better measure of spread. So from this perspective, "convenience" often is the right answer.

On the other hand, as a theorist, you would probably want to develop the theory along the most fruitful route, so you would be stupid to ignore the variance in any case. Its usefulness as a measure of spread then is less important, and its overall properties are rather more important. From this perspective, "convenience" isn't really the right answer, since it doesn't convey the intrinsic value it has by virtue of the excellent theory surrounding it.

Answer by bondmatt for why is it so cool to square numbers? (in terms of finding the standard deviation)

2010-07-30T01:22:20Z

Population variance, E([X-E(X)]^2), can quite easily be estimated in an unbiased manner by the sample variance, (n-1)^{-1}\sum (X_i-\bar{X})^2, where the sum is from i=1 to i=n. "Unbiased" meaning that if the X_i are i.i.d. copies of X, then the expectation of the sample variance (by a straightforward direct computation) is the population variance.

As far as I know, you can't do anything at all for absolute values. For even powers larger than 2, a more complicated unbiased estimator could be constructed. Two is the simplest case, and plenty of interest happens - the central limit theorem, for starters. I'm not sure to what extent the L^2 theory generalizes to L^2k theory, but to my knowledge no compelling reason to consider it exists. Perhaps with higher k, the large deviations are weighted even more heavily, and particular applications may benefit from their study. However, the usual variance is a good starting point. There are conditions under which you can recover a random variable X if you know all of its moments, E(X^p) for p = 1, 2, 3, .... From this perspective, one might consider the variance, E(X^2)-E(X)^2, a function of the first two moments, for the same reason that one may look at a second-degree Taylor approximation of sin - it's the simplest, and you can look at other terms if you need to know more.

Answer by david for why is it so cool to square numbers? (in terms of finding the standard deviation)

2011-09-10T10:16:07Z

I think that is quite obvious.

Firstly, the sampling standard deviation measures the dispersion of the data in the samples.

You square the difference is to reflect how much the actually data is away from the sample mean. If you do not square them, some differences may cancel out each other e.g. -1, +1, in which case the dispersion cannot be reflected.

You square root it is for matching the dimension