Why doesn't Stein effect happen for multinomial distributions? - MathOverflow most recent 30 from http://mathoverflow.net 2013-05-23T21:26:51Z http://mathoverflow.net/feeds/question/39029 http://www.creativecommons.org/licenses/by-nc/2.5/rdf http://mathoverflow.net/questions/39029/why-doesnt-stein-effect-happen-for-multinomial-distributions Why doesn't Stein effect happen for multinomial distributions? Yaroslav Bulatov 2010-09-16T22:20:03Z 2010-09-17T06:24:19Z <p><a href="http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.34.6043" rel="nofollow">(Medeen, et all, 1998)"</a> show that Maximum Likelihood estimate is admissible for multinomial distribution under squared error. On other hand, James and Stein showed that arithmetic average is not an admissible estimator of Gaussian location parameter in 3 dimensions. But since maximum likelihood estimates of multinomial parameters are averages of observed counts, which become normally distributed for large sample sizes, why doesn't Stein effect happen here? </p> <p>$\hat{p}$ is an inadmissible estimator of $\theta$ if there's an estimator that is no worse for every $\theta$ and better for at least one</p> http://mathoverflow.net/questions/39029/why-doesnt-stein-effect-happen-for-multinomial-distributions/39040#39040 Answer by R Hahn for Why doesn't Stein effect happen for multinomial distributions? R Hahn 2010-09-17T00:24:46Z 2010-09-17T06:24:19Z <p>This is not an answer, but maybe worth thinking about (and I cannot yet leave comments). My intuition about the Stein phenomenon is that while the individual coordinates of the Gaussian random variable are independent, the loss function involves all of the location parameters jointly. Stein type estimators take this into account and by doing so outperform the MLE, making it inadmissible. </p> <p>In the case of the multinomial parameters, they inherently have dependence via the sum-to-one constraint as a probability vector and you take this into account when averaging over possible parameter values. So a question related to yours, which may shed some light on it, is whether or not the MLE is admissible for a Gaussian location vector $\mu$ under the restriction that $\|\mu\| = c$ for some positive constant $c$. </p> <p>UPDATE: "Admissibility and complete class results for the multinomial estimation problem...", Ighodaro, Thomas &amp; Brown (Journal of Mult. Analysis '82) shows the MLE for the multinomial parameter becomes inadmissible if you remove the vertices of the simplex from the action space. It is a property of the risk behavior of the MLE at these extremal points that makes it admissible, then. Since the corresponding Gaussian problem has no such extremal points, this may constitute an explanation to your question.</p>