Consider probability distribution which is unimodal, symmetric and mean value exists. (Of course due to symmetry mean will coincide with mode (position of the maximum)).

Question Is sample mean always the "best" estimate of the mean or may be median, whatever can be better ?

Situation: assume the "tails" are heavy, (but still mean exists), so "sample-mean" will not be very stable, because big deviations will occur with big probability and hence mean will deviate of the "ideal" value. Median estimate seems to be more stable - if big deviations happens rarely median will not be affected by this.

So may be one can "cook up" something in the middle between mean and median - just filter out values which are at the ends and take average of the others or something like that...

  • $\begingroup$ What is your definition of "best" estimate, and what is the estimation setting - Bayesian or not? In the case of Bayesian MMSE estimation the answer is clearly no, as in general the optimal estimator (conditional expectation) in not linear in the measurement and therefore can't equal the sample mean. $\endgroup$ – Yair Carmon Oct 25 '12 at 9:34
  • $\begingroup$ @Yair Carmon you are right, "best" should be specified. What can be said with different choices of "best criteria" a) MMSE b) unbiased estimate with variance satisfying CramerRao bound ? $\endgroup$ – Alexander Chervov Oct 25 '12 at 9:55
  • $\begingroup$ Why "the answer is clearly no" for MMSE ? Yes conditional expectation is not linear - but may be due to the condition of symmetry and unimodality it will always coincide with sample mean - can you give counter-example ? $\endgroup$ – Alexander Chervov Oct 25 '12 at 9:59

First consider a non-bayesian setting in which $X_1,X_2,...,X_n$ are i.i.d. with a Laplace distribution with unknown mean $\mu$. Clearly, the maximum-likelihood estimator here is the sample median and not the sample mean. While in general the ML estimator is non-optimal in the sense that it will not yield the minimum MSE for any value of $\mu$, in the limit $n\to\infty$ it becomes optimal and attains the Cramer-Rao lower bound. We may therefore conclude that in this case and for sufficiently large $n$, the sample median will dominate the sample mean.

Regarding the Bayesian case, let's now assume the $\mu$ is uniformly distributed on $[-1,1]$ and $n=1$. In this case the "sample mean" is simply $X_1$, which is a linear estimator. Clearly, $E[\mu|X_1]$ is a non-linear function of $X_1$ and cannot therefore be the sample mean. However, by simple symmetry considerations one can see that if $X_i=\mu+\tilde{X}_i$ where $\tilde{X}_i$ have zero-mean, and are uncorrelated with each other and with $\mu$, then the sample mean $\frac{1}{n}\sum _{i=1}^n X_i$ is the optimal unbiased linear estimator of $\mu$ given $X_1,...,X_n$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.