# Why does sample standard deviation underestimate population standard deviation? [closed]

I am aware of Jensen's inequality where, given the concave square root function, the mean of the square root is lesser than the square root of the mean.

However, I cannot figure out why the square root of the unbiased estimator for population variance will be less than the true population standard deviation. A lot of text on the web state Jensen's inequality to explain that it is indeed smaller without actually explaining why.

Am I right to think that since the square root of the unbiased estimator is lesser so it must somehow relate to the "mean of the square root" while the population standard deviation somehow relates to "square root of the mean" (to use the terms from the Jensen's inequality example)?

Responding to Timothy Chow:

I understand, in the description of your procedure, how Jensen's inequality comes into play in both cases. It is apparent in your procedure because you are taking multiple samplings from the population.

I suppose all descriptions of estimating population variance (or standard deviation) assume that one can take multiple samplings from the population for analysis. I am interested in estimating population variables using one single sampling from the population. I am aware that one single sampling from the population is extremely unreliable but in certain situations, once is all you get. The background of my question is in financial applications.

I re-frame my question as follows:

In estimating population standard deviation from a single sampling of data, will the square root of the unbiased estimator for population variance underestimate population standard deviation? If it does, why is this so, since there is no averaging of unbiased estimators for population variance across multiple samplings?

-

## closed as off topic by Timothy Chow, Qiaochu Yuan, Yemon Choi, Douglas Zare, Andy PutmanFeb 8 '12 at 4:39

Questions on MathOverflow are expected to relate to research level mathematics within the scope defined by the community. Consider editing the question or leaving comments for improvement if you believe the question can be reworded to fit within the scope. Read more about reopening questions here.If this question can be reworded to fit the rules in the help center, please edit the question.

This is really too elementary a question for MO but I'm answering it below anyway. – Timothy Chow Feb 7 '12 at 3:19
To respond to some of your comments to Timothy Chow's answer: MathOverflow is not a substitute for a course in statistics. Which is not to say that these things are easy if one hasn't seen them before; but MathOverflow is not the place to get these things explained. – Yemon Choi Feb 8 '12 at 5:27
@JohnC, I think you would get better answers if you post your question at stats.stackexchange.com. – Joel Reyes Noche Feb 8 '12 at 10:47
Thanks Joel appreciate the link – JohnC Feb 8 '12 at 16:21
JohnC, in response to your re-framed question: If you take only one sample, then you can't say for sure whether it will be an underestimate or not. Sometimes it will; sometimes it won't. Just as if you roll a die once, you can't say for sure whether it will come up less than 4.5 or greater than 4.5. But you can say that the value you get will tend to be less than 4.5 because the average value of a die roll is 3.5. That is why averages come into play even if you are taking just one sample and trying to predict what will happen. – Timothy Chow Feb 9 '12 at 3:17

## 1 Answer

Your intuition is basically correct. It's perhaps easiest to see in the special case where your population is finite and you can explicitly enumerate all possible samples from the population. Imagine doing so, and computing the unbiased estimator for the variance in each case. Label these estimates $v_1, v_2, \ldots, v_n$. By definition of "unbiased," the mean of these numbers is the population variance, so the square root of their mean $\sqrt{(v_1 + \cdots + v_n)/n}$ is the population standard deviation. But now suppose you take the square root of the unbiased estimator of the variance as your estimate of the standard deviation. Then you're computing $\sqrt{v_1}, \sqrt{v_2}, \ldots, \sqrt{v_n}$. The mean of these numbers $(\sqrt{v_1} +\cdots + \sqrt{v_n})/n$ is going to be less than the $\sqrt{(v_1 + \cdots + v_n)/n}$, by Jensen's inequality for example. That is, the square root of the unbiased estimator will, on average, be smaller than the number you're really interested in.

For the general case you'll have to fuss with integrals but the logic of the argument is identical.

-
Timothy, please see my edits to the question in response to your proposed answer. – JohnC Feb 7 '12 at 17:14
"That is, the square root of the unbiased estimator will, on average, be smaller than the number you're really interested in." I am confused by this sentence. Is the "square root of the unbiased estimator" the value you described in case 1 or in case 2? Also, going by your logic, and assuming I can take a huge number of independent samplings from the population, would I not then get a very good estimate of population std dev by simply following the procedure you described in case 1? Why is it "common knowledge" then that the sample std dev underestimates population std dev? – JohnC Feb 8 '12 at 3:24
JohnC: It is "common knowledge" that the sample std dev underestimates the population std dev because most of those people who care about this subject have taken the trouble to understand the argument that Timothy Chow just gave you. – Steven Landsburg Feb 8 '12 at 3:47
Believe me, I am trying to makes sense of all information posted here. Would you help me in answering my questions? – JohnC Feb 8 '12 at 4:18
Timothy: Is "square root of the unbiased estimator" referring to $\sqrt{(v_1 + \cdots + v_n)/n}$ or $(\sqrt{v_1} +\cdots + \sqrt{v_n})/n$? You mentioned "square root of the unbiased estimator will, on average, be smaller" but also that $\sqrt{(v_1 + \cdots + v_n)/n}$ is greater than $(\sqrt{v_1} +\cdots + \sqrt{v_n})/n$. – JohnC Feb 8 '12 at 4:31