# Proof of no unbiased estimation of standard deviation.

It is well known that for iid random variables $X_1, \ldots, X_n$ with variance $\sigma^2$ that

$$\frac{1}{n-1} \sum_{i=1}^n (X_i - \overline X)^2$$

gives an unbiased estimator for $\sigma^2$, but

$$\sqrt{ \frac{1}{n-1} \sum_{i=1}^n (X_i - \overline X)^2 }$$

is not an unbiased estimator for $\sigma$.

Is there a way to formulate precisely and prove that there does not exist an unbiased estimator $f(X_1,\ldots,X_n)$ for $\sigma$ that works for all probability distributions?

Consider the case $n=2$. By scaling and translation we must have $f(x,y) = c |x-y|$ for some constant $c$. But for e.g. a Bernoulli distribution $E[c|X_1 - X_2|] = c p(1-p)$, and this can't be $\sigma = \sqrt{p(1-p)}$ for all $p$.
• Similarly for $n=3$: we must have $f(0,0,0) = f(1,1,1) = 0$ and $f(x,y,z) = c$ for all other $(x,y,z) \in \{0,1\}^3$. Then for a Bernoulli distibution $E[f(X_1,X_2,X_3)] = 3 c p (1-p)$ which can't be $\sqrt{p(1-p)}$. – Robert Israel Jun 27 '12 at 0:05
• Similarly for any $n$: let $f(X_1,\ldots,X_n) = c_k$ if $k$ of the $X_i$ are $0$ and the other $n-k$ are $1$. We must have $c_0 = 0$. Now for a Bernoulli distribution with small $p$, $E[f(X_1,\ldots,X_n)] = n p c_1 + O(p^2)$, and this can't be $\sqrt{p(1-p)}$. – Robert Israel Jun 27 '12 at 0:24
• Can you explain in detail why $f(x,y) = c|x-y|$ is true? What exactly do you mean by "scaling and translation we must have $f(x,y) = c|x-y|$" – user16557 Jun 30 '12 at 6:52
• Actually that's not quite true: what would be true is that $f(x,y) + f(y,x) = c |x-y|$. But of course we can assume $f$ is symmetric in $x$ and $y$. The point is that $E[f(aX_1+b,aX_2+b)] = |a|E[f(X_1,X_2)]$ because $\sigma_{aX+b}=|a| \sigma_X$. – Robert Israel Jul 1 '12 at 6:14
• I'm having trouble understanding how the independence allows the variable $a$ to escape the function $f$. – user16557 Jul 3 '12 at 3:27
If the statistical model consists in assuming $X_1,\ldots,X_n$ to be iid Bernoulli with unknown success probablity $p\in[0,1]$, then a function $\kappa:[0,1]\rightarrow {\mathbb R}$ admits an unbiased estimator iff $\kappa$ is polynomial of degree at most $n$. ("Only if" is clear by writing down the expectation of any estimator, "if" follows by observing that $\prod_{i=1}^kX_i$ has expectation $p^k$ and using linearity.) Now $\sigma(p)=\sqrt{p(1-p)}$ is not polynomial, hence does not admit any unbiased estimator here.