0
$\begingroup$

I am aware of Jensen's inequality where, given the concave square root function, the mean of the square root is lesser than the square root of the mean.

However, I cannot figure out why the square root of the unbiased estimator for population variance will be less than the true population standard deviation. A lot of text on the web state Jensen's inequality to explain that it is indeed smaller without actually explaining why.

Am I right to think that since the square root of the unbiased estimator is lesser so it must somehow relate to the "mean of the square root" while the population standard deviation somehow relates to "square root of the mean" (to use the terms from the Jensen's inequality example)?

Responding to Timothy Chow:

I understand, in the description of your procedure, how Jensen's inequality comes into play in both cases. It is apparent in your procedure because you are taking multiple samplings from the population.

I suppose all descriptions of estimating population variance (or standard deviation) assume that one can take multiple samplings from the population for analysis. I am interested in estimating population variables using one single sampling from the population. I am aware that one single sampling from the population is extremely unreliable but in certain situations, once is all you get. The background of my question is in financial applications.

I re-frame my question as follows:

In estimating population standard deviation from a single sampling of data, will the square root of the unbiased estimator for population variance underestimate population standard deviation? If it does, why is this so, since there is no averaging of unbiased estimators for population variance across multiple samplings?

$\endgroup$
4
  • 2
    $\begingroup$ This is really too elementary a question for MO but I'm answering it below anyway. $\endgroup$ Feb 7, 2012 at 3:19
  • $\begingroup$ To respond to some of your comments to Timothy Chow's answer: MathOverflow is not a substitute for a course in statistics. Which is not to say that these things are easy if one hasn't seen them before; but MathOverflow is not the place to get these things explained. $\endgroup$
    – Yemon Choi
    Feb 8, 2012 at 5:27
  • 2
    $\begingroup$ @JohnC, I think you would get better answers if you post your question at stats.stackexchange.com. $\endgroup$
    – JRN
    Feb 8, 2012 at 10:47
  • $\begingroup$ JohnC, in response to your re-framed question: If you take only one sample, then you can't say for sure whether it will be an underestimate or not. Sometimes it will; sometimes it won't. Just as if you roll a die once, you can't say for sure whether it will come up less than 4.5 or greater than 4.5. But you can say that the value you get will tend to be less than 4.5 because the average value of a die roll is 3.5. That is why averages come into play even if you are taking just one sample and trying to predict what will happen. $\endgroup$ Feb 9, 2012 at 3:17

1 Answer 1

3
$\begingroup$

Your intuition is basically correct. It's perhaps easiest to see in the special case where your population is finite and you can explicitly enumerate all possible samples from the population. Imagine doing so, and computing the unbiased estimator for the variance in each case. Label these estimates $v_1, v_2, \ldots, v_n$. By definition of "unbiased," the mean of these numbers is the population variance, so the square root of their mean $\sqrt{(v_1 + \cdots + v_n)/n}$ is the population standard deviation. But now suppose you take the square root of the unbiased estimator of the variance as your estimate of the standard deviation. Then you're computing $\sqrt{v_1}, \sqrt{v_2}, \ldots, \sqrt{v_n}$. The mean of these numbers $(\sqrt{v_1} +\cdots + \sqrt{v_n})/n $ is going to be less than the $\sqrt{(v_1 + \cdots + v_n)/n}$, by Jensen's inequality for example. That is, the square root of the unbiased estimator will, on average, be smaller than the number you're really interested in.

For the general case you'll have to fuss with integrals but the logic of the argument is identical.

$\endgroup$
6
  • $\begingroup$ Timothy, please see my edits to the question in response to your proposed answer. $\endgroup$
    – JohnC
    Feb 7, 2012 at 17:14
  • $\begingroup$ "That is, the square root of the unbiased estimator will, on average, be smaller than the number you're really interested in." I am confused by this sentence. Is the "square root of the unbiased estimator" the value you described in case 1 or in case 2? Also, going by your logic, and assuming I can take a huge number of independent samplings from the population, would I not then get a very good estimate of population std dev by simply following the procedure you described in case 1? Why is it "common knowledge" then that the sample std dev underestimates population std dev? $\endgroup$
    – JohnC
    Feb 8, 2012 at 3:24
  • $\begingroup$ JohnC: It is "common knowledge" that the sample std dev underestimates the population std dev because most of those people who care about this subject have taken the trouble to understand the argument that Timothy Chow just gave you. $\endgroup$ Feb 8, 2012 at 3:47
  • $\begingroup$ Believe me, I am trying to makes sense of all information posted here. Would you help me in answering my questions? $\endgroup$
    – JohnC
    Feb 8, 2012 at 4:18
  • $\begingroup$ Timothy: Is "square root of the unbiased estimator" referring to $\sqrt{(v_1 + \cdots + v_n)/n}$ or $(\sqrt{v_1} +\cdots + \sqrt{v_n})/n$? You mentioned "square root of the unbiased estimator will, on average, be smaller" but also that $\sqrt{(v_1 + \cdots + v_n)/n}$ is greater than $(\sqrt{v_1} +\cdots + \sqrt{v_n})/n$. $\endgroup$
    – JohnC
    Feb 8, 2012 at 4:31

Not the answer you're looking for? Browse other questions tagged or ask your own question.