Take the 2-minute tour ×
MathOverflow is a question and answer site for professional mathematicians. It's 100% free, no registration required.

(Medeen, et all, 1998)" show that Maximum Likelihood estimate is admissible for multinomial distribution under squared error. On other hand, James and Stein showed that arithmetic average is not an admissible estimator of Gaussian location parameter in 3 dimensions. But since maximum likelihood estimates of multinomial parameters are averages of observed counts, which become normally distributed for large sample sizes, why doesn't Stein effect happen here?

$\hat{p}$ is an inadmissible estimator of $\theta$ if there's an estimator that is no worse for every $\theta$ and better for at least one

share|improve this question
    
Minor comment: I believe that was James and Stein, not "James Stein". –  Mark Meckes Sep 17 '10 at 1:01
add comment

1 Answer 1

up vote 4 down vote accepted

This is not an answer, but maybe worth thinking about (and I cannot yet leave comments). My intuition about the Stein phenomenon is that while the individual coordinates of the Gaussian random variable are independent, the loss function involves all of the location parameters jointly. Stein type estimators take this into account and by doing so outperform the MLE, making it inadmissible.

In the case of the multinomial parameters, they inherently have dependence via the sum-to-one constraint as a probability vector and you take this into account when averaging over possible parameter values. So a question related to yours, which may shed some light on it, is whether or not the MLE is admissible for a Gaussian location vector $\mu$ under the restriction that $\|\mu\| = c$ for some positive constant $c$.

UPDATE: "Admissibility and complete class results for the multinomial estimation problem...", Ighodaro, Thomas & Brown (Journal of Mult. Analysis '82) shows the MLE for the multinomial parameter becomes inadmissible if you remove the vertices of the simplex from the action space. It is a property of the risk behavior of the MLE at these extremal points that makes it admissible, then. Since the corresponding Gaussian problem has no such extremal points, this may constitute an explanation to your question.

share|improve this answer
    
I think normalization issue is minor because you can just drop the last parameter, and measure admissibility on the reduced parameter vector. To make it more in line with Gaussian example, let your random variable be an average of k observations with last component dropped. For large k it becomes distributed as a Gaussian. Then MLE estimate of the mean of your distribution (first d-1 components) is an arithmetic average of a sample of (approximately) Gaussian-distributed random variables –  Yaroslav Bulatov Sep 17 '10 at 4:58
    
It isn't the normalization per se, it's the restriction of the action space. I've edited my answer to clarify this and included a reference. –  R Hahn Sep 17 '10 at 6:25
    
Nice find! So I guess Stein effect does happen there. –  Yaroslav Bulatov Sep 18 '10 at 19:08
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.