3
$\begingroup$

(Medeen, et all, 1998)" show that Maximum Likelihood estimate is admissible for multinomial distribution under squared error. On other hand, James and Stein showed that arithmetic average is not an admissible estimator of Gaussian location parameter in 3 dimensions. But since maximum likelihood estimates of multinomial parameters are averages of observed counts, which become normally distributed for large sample sizes, why doesn't Stein effect happen here?

$\hat{p}$ is an inadmissible estimator of $\theta$ if there's an estimator that is no worse for every $\theta$ and better for at least one

Improve this question
$\endgroup$
1
  • $\begingroup$ Minor comment: I believe that was James and Stein, not "James Stein". $\endgroup$
    – Mark Meckes
    Commented Sep 17, 2010 at 1:01

1 Answer 1

Reset to default
6
$\begingroup$

This is not an answer, but maybe worth thinking about (and I cannot yet leave comments). My intuition about the Stein phenomenon is that while the individual coordinates of the Gaussian random variable are independent, the loss function involves all of the location parameters jointly. Stein type estimators take this into account and by doing so outperform the MLE, making it inadmissible.

In the case of the multinomial parameters, they inherently have dependence via the sum-to-one constraint as a probability vector and you take this into account when averaging over possible parameter values. So a question related to yours, which may shed some light on it, is whether or not the MLE is admissible for a Gaussian location vector $\mu$ under the restriction that $\|\mu\| = c$ for some positive constant $c$.

UPDATE: "Admissibility and complete class results for the multinomial estimation problem...", Ighodaro, Thomas & Brown (Journal of Mult. Analysis '82) shows the MLE for the multinomial parameter becomes inadmissible if you remove the vertices of the simplex from the action space. It is a property of the risk behavior of the MLE at these extremal points that makes it admissible, then. Since the corresponding Gaussian problem has no such extremal points, this may constitute an explanation to your question.

Improve this answer
$\endgroup$
3
  • $\begingroup$ I think normalization issue is minor because you can just drop the last parameter, and measure admissibility on the reduced parameter vector. To make it more in line with Gaussian example, let your random variable be an average of k observations with last component dropped. For large k it becomes distributed as a Gaussian. Then MLE estimate of the mean of your distribution (first d-1 components) is an arithmetic average of a sample of (approximately) Gaussian-distributed random variables $\endgroup$
    – Yaroslav Bulatov
    Commented Sep 17, 2010 at 4:58
  • $\begingroup$ It isn't the normalization per se, it's the restriction of the action space. I've edited my answer to clarify this and included a reference. $\endgroup$
    – R Hahn
    Commented Sep 17, 2010 at 6:25
  • $\begingroup$ Nice find! So I guess Stein effect does happen there. $\endgroup$
    – Yaroslav Bulatov
    Commented Sep 18, 2010 at 19:08

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .