Take the 2-minute tour ×
MathOverflow is a question and answer site for professional mathematicians. It's 100% free, no registration required.

It is well known that for iid random variables $X_1, \ldots, X_n$ with variance $\sigma^2$ that

$$\frac{1}{n-1} \sum_{i=1}^n (X_i - \overline X)^2$$

gives an unbiased estimator for $\sigma^2$, but

$$\sqrt{ \frac{1}{n-1} \sum_{i=1}^n (X_i - \overline X)^2 }$$

is not an unbiased estimator for $\sigma$.

Is there a way to formulate precisely and prove that there does not exist an unbiased estimator $f(X_1,\ldots,X_n)$ for $\sigma$ that works for all probability distributions?

share|improve this question
add comment

1 Answer

Consider the case $n=2$. By scaling and translation we must have $f(x,y) = c |x-y|$ for some constant $c$. But for e.g. a Bernoulli distribution $E[c|X_1 - X_2|] = c p(1-p)$, and this can't be $\sigma = \sqrt{p(1-p)}$ for all $p$.

share|improve this answer
    
Similarly for $n=3$: we must have $f(0,0,0) = f(1,1,1) = 0$ and $f(x,y,z) = c$ for all other $(x,y,z) \in \{0,1\}^3$. Then for a Bernoulli distibution $E[f(X_1,X_2,X_3)] = 3 c p (1-p)$ which can't be $\sqrt{p(1-p)}$. –  Robert Israel Jun 27 '12 at 0:05
    
Similarly for any $n$: let $f(X_1,\ldots,X_n) = c_k$ if $k$ of the $X_i$ are $0$ and the other $n-k$ are $1$. We must have $c_0 = 0$. Now for a Bernoulli distribution with small $p$, $E[f(X_1,\ldots,X_n)] = n p c_1 + O(p^2)$, and this can't be $\sqrt{p(1-p)}$. –  Robert Israel Jun 27 '12 at 0:24
    
Can you explain in detail why $f(x,y) = c|x-y|$ is true? What exactly do you mean by "scaling and translation we must have $f(x,y) = c|x-y|$" –  user16557 Jun 30 '12 at 6:52
    
Actually that's not quite true: what would be true is that $f(x,y) + f(y,x) = c |x-y|$. But of course we can assume $f$ is symmetric in $x$ and $y$. The point is that $E[f(aX_1+b,aX_2+b)] = |a|E[f(X_1,X_2)]$ because $\sigma_{aX+b}=|a| \sigma_X$. –  Robert Israel Jul 1 '12 at 6:14
    
How do you conclude $E[f(aX1+b,aX2+b)]=|a|E[f(X1,X2)]$? What justifies pulling $a$ outside of $f$ and outside of the expectation? –  user16557 Jul 2 '12 at 3:03
show 4 more comments

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.