Question

Consider probability distribution which is unimodal, symmetric and mean value exists. (Of course due to symmetry mean will coincide with mode (position of the maximum)).

Question Is sample mean always the "best" estimate of the mean or may be median, whatever can be better ?

Situation: assume the "tails" are heavy, (but still mean exists), so "sample-mean" will not be very stable, because big deviations will occur with big probability and hence mean will deviate of the "ideal" value. Median estimate seems to be more stable - if big deviations happens rarely median will not be affected by this.

So may be one can "cook up" something in the middle between mean and median - just filter out values which are at the ends and take average of the others or something like that...

Answer 1

First consider a non-bayesian setting in which $X_1,X_2,...,X_n$ are i.i.d. with a Laplace distribution with unknown mean $\mu$. Clearly, the maximum-likelihood estimator here is the sample median and not the sample mean. While in general the ML estimator is non-optimal in the sense that it will not yield the minimum MSE for any value of $\mu$, in the limit $n\to\infty$ it becomes optimal and attains the Cramer-Rao lower bound. We may therefore conclude that in this case and for sufficiently large $n$, the sample median will dominate the sample mean.

Regarding the Bayesian case, let's now assume the $\mu$ is uniformly distributed on $[-1,1]$ and $n=1$. In this case the "sample mean" is simply $X_1$, which is a linear estimator. Clearly, $E[\mu|X_1]$ is a non-linear function of $X_1$ and cannot therefore be the sample mean. However, by simple symmetry considerations one can see that if $X_i=\mu+\tilde{X}_i$ where $\tilde{X}_i$ have zero-mean, and are uncorrelated with each other and with $\mu$, then the sample mean $\frac{1}{n}\sum _{i=1}^n X_i$ is the optimal unbiased linear estimator of $\mu$ given $X_1,...,X_n$.