4
$\begingroup$

It is well known that for iid random variables $X_1, \ldots, X_n$ with variance $\sigma^2$ that

$$\frac{1}{n-1} \sum_{i=1}^n (X_i - \overline X)^2$$

gives an unbiased estimator for $\sigma^2$, but

$$\sqrt{ \frac{1}{n-1} \sum_{i=1}^n (X_i - \overline X)^2 }$$

is not an unbiased estimator for $\sigma$.

Is there a way to formulate precisely and prove that there does not exist an unbiased estimator $f(X_1,\ldots,X_n)$ for $\sigma$ that works for all probability distributions?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
7
$\begingroup$

Consider the case $n=2$. By scaling and translation we must have $f(x,y) = c |x-y|$ for some constant $c$. But for e.g. a Bernoulli distribution $E[c|X_1 - X_2|] = c p(1-p)$, and this can't be $\sigma = \sqrt{p(1-p)}$ for all $p$.

Improve this answer
$\endgroup$
10
  • $\begingroup$ Similarly for $n=3$: we must have $f(0,0,0) = f(1,1,1) = 0$ and $f(x,y,z) = c$ for all other $(x,y,z) \in \{0,1\}^3$. Then for a Bernoulli distibution $E[f(X_1,X_2,X_3)] = 3 c p (1-p)$ which can't be $\sqrt{p(1-p)}$. $\endgroup$
    – Robert Israel
    Commented Jun 27, 2012 at 0:05
  • $\begingroup$ Similarly for any $n$: let $f(X_1,\ldots,X_n) = c_k$ if $k$ of the $X_i$ are $0$ and the other $n-k$ are $1$. We must have $c_0 = 0$. Now for a Bernoulli distribution with small $p$, $E[f(X_1,\ldots,X_n)] = n p c_1 + O(p^2)$, and this can't be $\sqrt{p(1-p)}$. $\endgroup$
    – Robert Israel
    Commented Jun 27, 2012 at 0:24
  • 1
    $\begingroup$ Can you explain in detail why $f(x,y) = c|x-y|$ is true? What exactly do you mean by "scaling and translation we must have $f(x,y) = c|x-y|$" $\endgroup$
    – user16557
    Commented Jun 30, 2012 at 6:52
  • 1
    $\begingroup$ Actually that's not quite true: what would be true is that $f(x,y) + f(y,x) = c |x-y|$. But of course we can assume $f$ is symmetric in $x$ and $y$. The point is that $E[f(aX_1+b,aX_2+b)] = |a|E[f(X_1,X_2)]$ because $\sigma_{aX+b}=|a| \sigma_X$. $\endgroup$
    – Robert Israel
    Commented Jul 1, 2012 at 6:14
  • 1
    $\begingroup$ I'm having trouble understanding how the independence allows the variable $a$ to escape the function $f$. $\endgroup$
    – user16557
    Commented Jul 3, 2012 at 3:27
4
$\begingroup$

If the statistical model consists in assuming $X_1,\ldots,X_n$ to be iid Bernoulli with unknown success probablity $p\in[0,1]$, then a function $\kappa:[0,1]\rightarrow {\mathbb R}$ admits an unbiased estimator iff $\kappa$ is polynomial of degree at most $n$. ("Only if" is clear by writing down the expectation of any estimator, "if" follows by observing that $\prod_{i=1}^kX_i$ has expectation $p^k$ and using linearity.) Now $\sigma(p)=\sqrt{p(1-p)}$ is not polynomial, hence does not admit any unbiased estimator here.

The above is known from sufficiently good textbooks of mathematical statistics, see for example Pfanzagl (1994), Parametric Statistical Theory, page 72, where the equivalent (by a sufficiency reduction) case of one binomial observation is treated. Pfanzagl credits a paper of Kolmogorov (1950), Unbiased estimates, who however cites Halmos (1946), The theory of unbiased estimation, which contains at least the basic idea.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .