Take the 2-minute tour ×
MathOverflow is a question and answer site for professional mathematicians. It's 100% free, no registration required.

It is well known that for iid random variables $X_1, \ldots, X_n$ with variance $\sigma^2$ that

$$\frac{1}{n-1} \sum_{i=1}^n (X_i - \overline X)^2$$

gives an unbiased estimator for $\sigma^2$, but

$$\sqrt{ \frac{1}{n-1} \sum_{i=1}^n (X_i - \overline X)^2 }$$

is not an unbiased estimator for $\sigma$.

Is there a way to formulate precisely and prove that there does not exist an unbiased estimator $f(X_1,\ldots,X_n)$ for $\sigma$ that works for all probability distributions?

share|improve this question

2 Answers 2

Consider the case $n=2$. By scaling and translation we must have $f(x,y) = c |x-y|$ for some constant $c$. But for e.g. a Bernoulli distribution $E[c|X_1 - X_2|] = c p(1-p)$, and this can't be $\sigma = \sqrt{p(1-p)}$ for all $p$.

share|improve this answer
    
Similarly for $n=3$: we must have $f(0,0,0) = f(1,1,1) = 0$ and $f(x,y,z) = c$ for all other $(x,y,z) \in \{0,1\}^3$. Then for a Bernoulli distibution $E[f(X_1,X_2,X_3)] = 3 c p (1-p)$ which can't be $\sqrt{p(1-p)}$. –  Robert Israel Jun 27 '12 at 0:05
    
Similarly for any $n$: let $f(X_1,\ldots,X_n) = c_k$ if $k$ of the $X_i$ are $0$ and the other $n-k$ are $1$. We must have $c_0 = 0$. Now for a Bernoulli distribution with small $p$, $E[f(X_1,\ldots,X_n)] = n p c_1 + O(p^2)$, and this can't be $\sqrt{p(1-p)}$. –  Robert Israel Jun 27 '12 at 0:24
    
Can you explain in detail why $f(x,y) = c|x-y|$ is true? What exactly do you mean by "scaling and translation we must have $f(x,y) = c|x-y|$" –  user16557 Jun 30 '12 at 6:52
1  
Actually that's not quite true: what would be true is that $f(x,y) + f(y,x) = c |x-y|$. But of course we can assume $f$ is symmetric in $x$ and $y$. The point is that $E[f(aX_1+b,aX_2+b)] = |a|E[f(X_1,X_2)]$ because $\sigma_{aX+b}=|a| \sigma_X$. –  Robert Israel Jul 1 '12 at 6:14
    
How do you conclude $E[f(aX1+b,aX2+b)]=|a|E[f(X1,X2)]$? What justifies pulling $a$ outside of $f$ and outside of the expectation? –  user16557 Jul 2 '12 at 3:03

If the statistical model consists in assuming $X_1,\ldots,X_n$ to be iid Bernoulli with unknown success probablity $p\in[0,1]$, then a function $\kappa:[0,1]\rightarrow {\mathbb R}$ admits an unbiased estimator iff $\kappa$ is polynomial of degree at most $n$. ("Only if" is clear by writing down the expectation of any estimator, "if" follows by observing that $\prod_{i=1}^kX_i$ has expectation $p^k$ and using linearity.) Now $\sigma(p)=\sqrt{p(1-p)}$ is not polynomial, hence does not admit any unbiased estimator here.

The above is known from sufficiently good textbooks of mathematical statistics, see for example Pfanzagl (1994), Parametric Statistical Theory, page 72, where the equivalent (by a sufficiency reduction) case of one binomial observation is treated. Pfanzagl credits a paper of Kolmogorov (1950), Unbiased estimates, who however cites Halmos (1946), The theory of unbiased estimation, which contains at least the basic idea.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.