4
$\begingroup$

It is well known that for iid random variables $X_1, \ldots, X_n$ with variance $\sigma^2$ that

$$\frac{1}{n-1} \sum_{i=1}^n (X_i - \overline X)^2$$

gives an unbiased estimator for $\sigma^2$, but

$$\sqrt{ \frac{1}{n-1} \sum_{i=1}^n (X_i - \overline X)^2 }$$

is not an unbiased estimator for $\sigma$.

Is there a way to formulate precisely and prove that there does not exist an unbiased estimator $f(X_1,\ldots,X_n)$ for $\sigma$ that works for all probability distributions?

$\endgroup$

2 Answers 2

6
$\begingroup$

Consider the case $n=2$. By scaling and translation we must have $f(x,y) = c |x-y|$ for some constant $c$. But for e.g. a Bernoulli distribution $E[c|X_1 - X_2|] = c p(1-p)$, and this can't be $\sigma = \sqrt{p(1-p)}$ for all $p$.

$\endgroup$
10
  • $\begingroup$ Similarly for $n=3$: we must have $f(0,0,0) = f(1,1,1) = 0$ and $f(x,y,z) = c$ for all other $(x,y,z) \in \{0,1\}^3$. Then for a Bernoulli distibution $E[f(X_1,X_2,X_3)] = 3 c p (1-p)$ which can't be $\sqrt{p(1-p)}$. $\endgroup$ Jun 27, 2012 at 0:05
  • $\begingroup$ Similarly for any $n$: let $f(X_1,\ldots,X_n) = c_k$ if $k$ of the $X_i$ are $0$ and the other $n-k$ are $1$. We must have $c_0 = 0$. Now for a Bernoulli distribution with small $p$, $E[f(X_1,\ldots,X_n)] = n p c_1 + O(p^2)$, and this can't be $\sqrt{p(1-p)}$. $\endgroup$ Jun 27, 2012 at 0:24
  • 1
    $\begingroup$ Can you explain in detail why $f(x,y) = c|x-y|$ is true? What exactly do you mean by "scaling and translation we must have $f(x,y) = c|x-y|$" $\endgroup$
    – user16557
    Jun 30, 2012 at 6:52
  • 1
    $\begingroup$ Actually that's not quite true: what would be true is that $f(x,y) + f(y,x) = c |x-y|$. But of course we can assume $f$ is symmetric in $x$ and $y$. The point is that $E[f(aX_1+b,aX_2+b)] = |a|E[f(X_1,X_2)]$ because $\sigma_{aX+b}=|a| \sigma_X$. $\endgroup$ Jul 1, 2012 at 6:14
  • 1
    $\begingroup$ I'm having trouble understanding how the independence allows the variable $a$ to escape the function $f$. $\endgroup$
    – user16557
    Jul 3, 2012 at 3:27
4
$\begingroup$

If the statistical model consists in assuming $X_1,\ldots,X_n$ to be iid Bernoulli with unknown success probablity $p\in[0,1]$, then a function $\kappa:[0,1]\rightarrow {\mathbb R}$ admits an unbiased estimator iff $\kappa$ is polynomial of degree at most $n$. ("Only if" is clear by writing down the expectation of any estimator, "if" follows by observing that $\prod_{i=1}^kX_i$ has expectation $p^k$ and using linearity.) Now $\sigma(p)=\sqrt{p(1-p)}$ is not polynomial, hence does not admit any unbiased estimator here.

The above is known from sufficiently good textbooks of mathematical statistics, see for example Pfanzagl (1994), Parametric Statistical Theory, page 72, where the equivalent (by a sufficiency reduction) case of one binomial observation is treated. Pfanzagl credits a paper of Kolmogorov (1950), Unbiased estimates, who however cites Halmos (1946), The theory of unbiased estimation, which contains at least the basic idea.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.